{
  "hf_id": "mistralai/Ministral-3-8B-Reasoning-2512",
  "meta": {
    "title": "Ministral-3-Reasoning",
    "slug": "ministral-3-reasoning",
    "provider": "Mistral AI",
    "description": "Ministral 3 Reasoning family (3B/8B/14B) with BF16 weights, vision support, and 256K context",
    "date_added": "2025-12-02",
    "date_updated": "2026-05-06",
    "tasks": [
      "multimodal"
    ],
    "related_recipes": [
      "mistralai/Ministral-3-14B-Instruct-2512",
      "mistralai/Mistral-Large-3-675B-Instruct-2512"
    ],
    "hardware": {
      "h200": "verified",
      "gb200": "verified",
      "mi300x": "verified"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:latest",
    "command": "vllm serve mistralai/Ministral-3-8B-Reasoning-2512 \\\n  --tokenizer_mode mistral \\\n  --config_format mistral \\\n  --load_format mistral \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser mistral \\\n  --reasoning-parser mistral",
    "argv": [
      "vllm",
      "serve",
      "mistralai/Ministral-3-8B-Reasoning-2512",
      "--tokenizer_mode",
      "mistral",
      "--config_format",
      "mistral",
      "--load_format",
      "mistral",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "mistral",
      "--reasoning-parser",
      "mistral"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:latest mistralai/Ministral-3-8B-Reasoning-2512 \\\n  --tokenizer_mode mistral \\\n  --config_format mistral \\\n  --load_format mistral \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser mistral \\\n  --reasoning-parser mistral",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:latest",
      "mistralai/Ministral-3-8B-Reasoning-2512",
      "--tokenizer_mode",
      "mistral",
      "--config_format",
      "mistral",
      "--load_format",
      "mistral",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "mistral",
      "--reasoning-parser",
      "mistral"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "multi_node_tp": "/mistralai/Ministral-3-8B-Reasoning-2512/strategies/multi_node_tp.json",
      "kv_store_centralized_mooncake": "/mistralai/Ministral-3-8B-Reasoning-2512/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/mistralai/Ministral-3-8B-Reasoning-2512/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/h200.json",
      "h100": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/h100.json",
      "b200": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/b200.json",
      "gb200": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/gb200.json",
      "b300": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/b300.json",
      "gb300": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/gb300.json",
      "mi300x": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/mi300x.json",
      "mi325x": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/mi325x.json",
      "mi350x": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/mi350x.json",
      "mi355x": "/mistralai/Ministral-3-8B-Reasoning-2512/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "mistralai/Ministral-3-8B-Reasoning-2512",
    "min_vllm_version": "0.11.0",
    "architecture": "dense",
    "parameter_count": "8B",
    "active_parameters": "8B",
    "context_length": 262144,
    "base_args": [
      "--tokenizer_mode",
      "mistral",
      "--config_format",
      "mistral",
      "--load_format",
      "mistral"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "Mistral tool-call parser with automatic tool choice",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "mistral"
      ]
    },
    "reasoning": {
      "description": "Mistral reasoning parser extracts <think>...</think> into message.reasoning",
      "args": [
        "--reasoning-parser",
        "mistral"
      ]
    },
    "text_only": {
      "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.",
      "args": [
        "--language-model-only"
      ]
    },
    "encoder_parallel": {
      "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.",
      "args": [
        "--mm-encoder-tp-mode",
        "data"
      ]
    }
  },
  "opt_in_features": [
    "text_only",
    "encoder_parallel"
  ],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 22,
      "description": "Native BF16 weights (8B)"
    },
    "3b": {
      "model_id": "mistralai/Ministral-3-3B-Reasoning-2512",
      "precision": "bf16",
      "vram_minimum_gb": 8,
      "description": "Smallest 3B variant with tied embeddings",
      "json": "/mistralai/Ministral-3-3B-Reasoning-2512.json"
    },
    "14b": {
      "model_id": "mistralai/Ministral-3-14B-Reasoning-2512",
      "precision": "bf16",
      "vram_minimum_gb": 32,
      "description": "Largest 14B variant; 2xH200 recommended for full context",
      "extra_args": [
        "--tensor-parallel-size",
        "2"
      ],
      "json": "/mistralai/Ministral-3-14B-Reasoning-2512.json"
    }
  },
  "kv_offload_support": {
    "offloading_cpu": "verified",
    "offloading_fs": "verified"
  },
  "hardware_overrides": {
    "amd": {
      "extra_args": [
        "--no-enable-prefix-caching"
      ],
      "extra_env": {
        "VLLM_ROCM_USE_AITER": "1",
        "SAFETENSORS_FAST_GPU": "1"
      }
    }
  },
  "guide": "## Overview\n\nMinistral-3 Reasoning comes with BF16 weights in 3 sizes:\n\n- 3B (tied embeddings)\n- 8B, 14B (independent embeddings/outputs)\n\nEach variant has vision support and 256K max context. On GB200, we observe significant\nspeed-ups with NVFP4 Marlin fallback for older GPUs.\n\n## Prerequisites\n\n- Hardware: 1x H200 (3B/8B), 2x H200 recommended for 14B with full context\n- vLLM >= 0.11.0\n\n### Install vLLM\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\n```\n\n## Launch command\n\n3B or 8B on 1x H200:\n\n```bash\nvllm serve mistralai/Ministral-3-8B-Reasoning-2512 \\\n  --tokenizer_mode mistral --config_format mistral --load_format mistral \\\n  --enable-auto-tool-choice --tool-call-parser mistral \\\n  --reasoning-parser mistral\n```\n\n14B on 2x H200:\n\n```bash\nvllm serve mistralai/Ministral-3-14B-Reasoning-2512 \\\n  --tensor-parallel-size 2 \\\n  --tokenizer_mode mistral --config_format mistral --load_format mistral \\\n  --enable-auto-tool-choice --tool-call-parser mistral \\\n  --reasoning-parser mistral\n```\n\nKey flags:\n- `enable-auto-tool-choice`: required for tool usage\n- `tool-call-parser mistral`: required for tool usage\n- `reasoning-parser mistral`: required to extract reasoning content\n\n## Client Usage\n\nStreaming reasoning + answer:\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\nstream = client.chat.completions.create(\n    model=\"mistralai/Ministral-3-8B-Reasoning-2512\",\n    messages=[{\"role\": \"user\", \"content\": \"Solve: use 2,5,6,3 to make 24.\"}],\n    stream=True, temperature=0.7, top_p=0.95, max_tokens=262144,\n)\nfor chunk in stream:\n    delta = chunk.choices[0].delta\n    # vLLM 0.11.0 uses reasoning_content; 0.11.1+ uses reasoning.\n    rc = getattr(delta, \"reasoning\", getattr(delta, \"reasoning_content\", None))\n    if rc:\n        print(rc, end=\"\", flush=True)\n    if delta.content:\n        print(delta.content, end=\"\", flush=True)\n```\n\n## Troubleshooting\n\n- OOM on 14B: use `--tensor-parallel-size 2` or lower `--max-model-len`.\n\n## References\n\n- [Ministral-3-8B-Reasoning](https://huggingface.co/mistralai/Ministral-3-8B-Reasoning-2512)\n- [Ministral-3-3B-Reasoning](https://huggingface.co/mistralai/Ministral-3-3B-Reasoning-2512)\n- [Ministral-3-14B-Reasoning](https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "389ffebe555705c85337025d5df108d0384d365e",
    "path": "recipes/models/mistralai/Ministral-3-8B-Reasoning-2512.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/mistralai/Ministral-3-8B-Reasoning-2512.yaml",
    "overlays": []
  }
}