{ "hf_id": "XiaomiMiMo/MiMo-V2-Flash", "meta": { "title": "MiMo-V2-Flash", "slug": "mimo-v2-flash", "provider": "MiMo (Xiaomi)", "description": "Xiaomi's MoE reasoning model (309B total / 15B active) with hybrid attention and MTP for fast agentic workflows", "date_added": "2025-12-20", "date_updated": "2026-10-08", "tasks": [ "text" ], "related_recipes": [], "hardware": { "h200": "verified", "mi300x": "verified", "mi325x": "verified", "mi355x": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve XiaomiMiMo/MiMo-V2-Flash \\\n --trust-remote-code \\\n --generation-config vllm \\\n --gpu-memory-utilization 0.9 \\\n --tensor-parallel-size 8 \\\n --tool-call-parser mimo \\\n --reasoning-parser qwen3", "argv": [ "vllm", "serve", "XiaomiMiMo/MiMo-V2-Flash", "--trust-remote-code", "--generation-config", "vllm", "--gpu-memory-utilization", "0.9", "--tensor-parallel-size", "8", "--tool-call-parser", "mimo", "--reasoning-parser", "qwen3" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest XiaomiMiMo/MiMo-V2-Flash \\\n --trust-remote-code \\\n --generation-config vllm \\\n --gpu-memory-utilization 0.9 \\\n --tensor-parallel-size 8 \\\n --tool-call-parser mimo \\\n --reasoning-parser qwen3", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "XiaomiMiMo/MiMo-V2-Flash", "--trust-remote-code", "--generation-config", "vllm", "--gpu-memory-utilization", "0.9", "--tensor-parallel-size", "8", "--tool-call-parser", "mimo", "--reasoning-parser", "qwen3" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "single_node_tep": "/XiaomiMiMo/MiMo-V2-Flash/strategies/single_node_tep.json", "multi_node_tp": "/XiaomiMiMo/MiMo-V2-Flash/strategies/multi_node_tp.json", "multi_node_tp_pp": "/XiaomiMiMo/MiMo-V2-Flash/strategies/multi_node_tp_pp.json", "multi_node_tep": "/XiaomiMiMo/MiMo-V2-Flash/strategies/multi_node_tep.json", "multi_node_dep": "/XiaomiMiMo/MiMo-V2-Flash/strategies/multi_node_dep.json", "pd_cluster": "/XiaomiMiMo/MiMo-V2-Flash/strategies/pd_cluster.json", "kv_store_centralized_mooncake": "/XiaomiMiMo/MiMo-V2-Flash/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/XiaomiMiMo/MiMo-V2-Flash/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/XiaomiMiMo/MiMo-V2-Flash/hw/h200.json", "h100": "/XiaomiMiMo/MiMo-V2-Flash/hw/h100.json", "b200": "/XiaomiMiMo/MiMo-V2-Flash/hw/b200.json", "gb200": "/XiaomiMiMo/MiMo-V2-Flash/hw/gb200.json", "b300": "/XiaomiMiMo/MiMo-V2-Flash/hw/b300.json", "gb300": "/XiaomiMiMo/MiMo-V2-Flash/hw/gb300.json", "mi300x": "/XiaomiMiMo/MiMo-V2-Flash/hw/mi300x.json", "mi325x": "/XiaomiMiMo/MiMo-V2-Flash/hw/mi325x.json", "mi350x": "/XiaomiMiMo/MiMo-V2-Flash/hw/mi350x.json", "mi355x": "/XiaomiMiMo/MiMo-V2-Flash/hw/mi355x.json" } }, "model": { "model_id": "XiaomiMiMo/MiMo-V2-Flash", "min_vllm_version": "0.20.0", "architecture": "moe", "parameter_count": "309B", "active_parameters": "15B", "context_length": 262144, "base_args": [ "--trust-remote-code", "--generation-config", "vllm" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "tool_calling": { "description": "MiMo tool-call parser", "args": [ "--tool-call-parser", "mimo" ] }, "reasoning": { "description": "Qwen3 reasoning parser", "args": [ "--reasoning-parser", "qwen3" ] } }, "opt_in_features": [], "variants": { "default": { "precision": "fp8", "vram_minimum_gb": 371, "description": "Native FP8 weights; 4x H200 recommended with TP4", "extra_args": [ "--tensor-parallel-size", "4", "--gpu-memory-utilization", "0.9" ] } }, "hardware_overrides": { "amd": { "extra_args": [], "extra_env": { "VLLM_ROCM_USE_AITER": "0" } } }, "guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and agentic\nworkflows, it features hybrid attention and Multi-Token Prediction (MTP) to reduce\ninference cost.\n\n## Prerequisites\n\n- Hardware: 4x H200 (TP4) or equivalent aggregate VRAM (~320 GB with FP8)\n- vLLM >= 0.20.0\n\n### Install vLLM (NVIDIA)\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install vllm --torch-backend auto\n```\n\n### Install vLLM (AMD ROCm MI300X/MI325X/MI355X)\n\n```bash\nuv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/\n```\n\n## Launch commands\n\nBasic TP4:\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2-Flash \\\n --host 0.0.0.0 --port 9001 --seed 1024 \\\n --served-model-name mimo_v2_flash \\\n --tensor-parallel-size 4 \\\n --trust-remote-code \\\n --gpu-memory-utilization 0.9 \\\n --generation-config vllm\n```\n\nWith tool calling + reasoning:\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2-Flash \\\n --tensor-parallel-size 4 --trust-remote-code --gpu-memory-utilization 0.9 \\\n --tool-call-parser mimo \\\n --reasoning-parser qwen3 \\\n --generation-config vllm \\\n --served-model-name mimo_v2_flash\n```\n\nDP + TP + EP:\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2-Flash \\\n --data-parallel-size 2 \\\n --tensor-parallel-size 4 \\\n --enable-expert-parallel \\\n --trust-remote-code \\\n --gpu-memory-utilization 0.9 \\\n --generation-config vllm \\\n --served-model-name mimo_v2_flash\n```\n\nAMD:\n\n```bash\nexport VLLM_ROCM_USE_AITER=0\nvllm serve XiaomiMiMo/MiMo-V2-Flash --tensor-parallel-size 4 \\\n --trust-remote-code --gpu-memory-utilization 0.9 --generation-config vllm\n```\n\nTunable flags:\n- `--max-model-len=65536` works well; max is 128K.\n- `--max-num-batched-tokens=32768` for prompt-heavy; 16K/8K for lower latency.\n- `--gpu-memory-utilization=0.95` to maximize KV cache.\n\n## Client Usage\n\n```bash\ncurl -X POST http://localhost:9001/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"mimo_v2_flash\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello MiMo!\"}],\n \"chat_template_kwargs\": {\"enable_thinking\": true}\n }'\n```\n\nSet `\"enable_thinking\": false` (or omit the kwargs) to disable thinking mode.\n\n## Benchmarking\n\n```bash\nvllm bench serve \\\n --model XiaomiMiMo/MiMo-V2-Flash \\\n --dataset-name random --random-input-len 8000 --random-output-len 1000 \\\n --request-rate 3 --num-prompts 1800 --ignore-eos\n```\n\n## Accuracy (GSM8K)\n\nReported 5-shot `exact_match`: flexible 0.9128, strict 0.9075.\n\n## References\n\n- [MiMo-V2-Flash on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/XiaomiMiMo/MiMo-V2-Flash.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/XiaomiMiMo/MiMo-V2-Flash.yaml", "overlays": [] } }