{
  "hf_id": "IFM/Jais-3-MoVA-36B-A4B",
  "meta": {
    "title": "Jais-3-MoVA-36B-A4B",
    "slug": "jais-3-mova-36b-a4b",
    "provider": "IFM",
    "description": "Sparse MoVA and MoE long-context model for research and production-style serving experiments",
    "date_added": "2026-10-05",
    "date_updated": "2026-10-05",
    "tasks": [
      "text"
    ],
    "performance_headline": "37.44B stored parameters including embeddings / 5.95B active parameters per token",
    "related_recipes": [
      "IFM/Jais-3-32B",
      "IFM/Jais-3-375B-A23B"
    ],
    "hardware": {
      "h200": "verified"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tep",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:latest",
    "command": "vllm serve IFM/Jais-3-MoVA-36B-A4B \\\n  --trust-remote-code \\\n  --chat-template-content-format string \\\n  --dtype bfloat16 \\\n  --enable-expert-parallel \\\n  --tensor-parallel-size 8 \\\n  --reasoning-parser k2_horizon \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser k2_horizon",
    "argv": [
      "vllm",
      "serve",
      "IFM/Jais-3-MoVA-36B-A4B",
      "--trust-remote-code",
      "--chat-template-content-format",
      "string",
      "--dtype",
      "bfloat16",
      "--enable-expert-parallel",
      "--tensor-parallel-size",
      "8",
      "--reasoning-parser",
      "k2_horizon",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "k2_horizon"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:latest IFM/Jais-3-MoVA-36B-A4B \\\n  --trust-remote-code \\\n  --chat-template-content-format string \\\n  --dtype bfloat16 \\\n  --enable-expert-parallel \\\n  --tensor-parallel-size 8 \\\n  --reasoning-parser k2_horizon \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser k2_horizon",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:latest",
      "IFM/Jais-3-MoVA-36B-A4B",
      "--trust-remote-code",
      "--chat-template-content-format",
      "string",
      "--dtype",
      "bfloat16",
      "--enable-expert-parallel",
      "--tensor-parallel-size",
      "8",
      "--reasoning-parser",
      "k2_horizon",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "k2_horizon"
    ],
    "strategy_spec": {
      "name": "single_node_tep",
      "deploy_type": "single_node",
      "display_name": "Tensor + Expert Parallel",
      "short_label": "TEP",
      "orientation": "balanced",
      "description": "Single-node TEP. TP splits dense layers and EP splits expert layers across local GPUs. TP must be set to GPU count to avoid OOM from replicated dense layers. For MoE models only.\n",
      "hardware_match": {
        "min_gpus": 2,
        "max_gpus": 8,
        "multi_node": false,
        "architecture": "moe"
      },
      "vllm_args": [
        "--enable-expert-parallel"
      ],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "multi_node_tep": "/IFM/Jais-3-MoVA-36B-A4B/strategies/multi_node_tep.json",
      "multi_node_tp_pp": "/IFM/Jais-3-MoVA-36B-A4B/strategies/multi_node_tp_pp.json",
      "kv_store_centralized_mooncake": "/IFM/Jais-3-MoVA-36B-A4B/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/IFM/Jais-3-MoVA-36B-A4B/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/IFM/Jais-3-MoVA-36B-A4B/hw/h200.json",
      "h100": "/IFM/Jais-3-MoVA-36B-A4B/hw/h100.json",
      "b200": "/IFM/Jais-3-MoVA-36B-A4B/hw/b200.json",
      "gb200": "/IFM/Jais-3-MoVA-36B-A4B/hw/gb200.json",
      "b300": "/IFM/Jais-3-MoVA-36B-A4B/hw/b300.json",
      "gb300": "/IFM/Jais-3-MoVA-36B-A4B/hw/gb300.json",
      "mi300x": "/IFM/Jais-3-MoVA-36B-A4B/hw/mi300x.json",
      "mi325x": "/IFM/Jais-3-MoVA-36B-A4B/hw/mi325x.json",
      "mi350x": "/IFM/Jais-3-MoVA-36B-A4B/hw/mi350x.json",
      "mi355x": "/IFM/Jais-3-MoVA-36B-A4B/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "IFM/Jais-3-MoVA-36B-A4B",
    "architecture": "moe",
    "parameter_count": "37.44B",
    "active_parameters": "5.95B",
    "context_length": 524288,
    "base_args": [
      "--trust-remote-code",
      "--chat-template-content-format",
      "string",
      "--dtype",
      "bfloat16",
      "--enable-expert-parallel"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "reasoning": {
      "description": "Extract Jais-3 reasoning into message.reasoning",
      "args": [
        "--reasoning-parser",
        "k2_horizon"
      ]
    },
    "tool_calling": {
      "description": "Enable automatic tool choice with the K2-Horizon parser",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "k2_horizon"
      ]
    }
  },
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 90,
      "description": "BF16 serving runtime with native vLLM execution"
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\nJais-3-MoVA-36B-A4B is a mixture-of-expert (MoE) model in the IFM Jais-3\nfamily. It combines MoVA attention and MoE layers, with 37.44B stored parameters\nincluding embeddings, 5.95B active parameters per token, and support for up\nto 524,288 tokens of context.\n\nJais-3 is an open-weight IFM model family built for transparent\nfoundation-model research, staged checkpoint analysis, and practical\ndeployment. The series spans compact dense models for local experimentation,\ndense mid-size and large models for high-quality research workloads, and\nmixture-of-expert (MoE) models such as Jais-3-MoVA-36B-A4B for higher-capacity\nserving.\n\n## Launch Commands\n\n```bash\nvllm serve IFM/Jais-3-MoVA-36B-A4B \\\n  --tensor-parallel-size 2 \\\n  --trust-remote-code \\\n  --chat-template-content-format string \\\n  --dtype bfloat16 \\\n  --enable-expert-parallel \\\n  --reasoning-parser k2_horizon \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser k2_horizon\n```\n\n## Client usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\", timeout=3600\n)\n\nresp = client.chat.completions.create(\n    model=\"IFM/Jais-3-MoVA-36B-A4B\",\n    messages=[{\"role\": \"user\", \"content\": \"لدينا عددان مجموعهما 50 وحاصل ضربهما 600. ما هما العددان؟ اشرح خطوات الحل.\"}],\n    temperature=1.0, top_p=0.95, max_tokens=2048,\n)\nprint(resp.choices[0].message.content)\n```\n\n### Thinking modes\n\nThe model supports selectable thinking through `chat_template_kwargs`, per\nrequest or server-wide via `--default-chat-template-kwargs`:\n\n- `{\"reasoning_effort\": \"high\"}` — full thinking (default).\n- `{\"reasoning_effort\": \"low\"}` — no thinking.\n\n## References\n\n- [IFM/Jais-3-MoVA-36B-A4B on Hugging Face](https://huggingface.co/IFM/Jais-3-MoVA-36B-A4B)\n- [Jais 3 Collection](https://huggingface.co/collections/IFM/jais-3)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "389ffebe555705c85337025d5df108d0384d365e",
    "path": "recipes/models/IFM/Jais-3-MoVA-36B-A4B.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/IFM/Jais-3-MoVA-36B-A4B.yaml",
    "overlays": []
  }
}