# XiaomiMiMo/MiMo-V2-Flash — vLLM serving recipe

> Xiaomi's MoE reasoning model (309B total / 15B active) with hybrid attention and MTP for fast agentic workflows

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (FP8) · features `tool_calling`, `reasoning`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/XiaomiMiMo/MiMo-V2-Flash
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/XiaomiMiMo/MiMo-V2-Flash.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `single_node_tep`, `multi_node_tp`, `multi_node_tp_pp`, `multi_node_tep`, `multi_node_dep`, `pd_cluster` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.20.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve XiaomiMiMo/MiMo-V2-Flash \
  --trust-remote-code \
  --generation-config vllm \
  --gpu-memory-utilization 0.9 \
  --tensor-parallel-size 8 \
  --tool-call-parser mimo \
  --reasoning-parser qwen3
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest XiaomiMiMo/MiMo-V2-Flash \
  --trust-remote-code \
  --generation-config vllm \
  --gpu-memory-utilization 0.9 \
  --tensor-parallel-size 8 \
  --tool-call-parser mimo \
  --reasoning-parser qwen3
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model
with 309B total parameters and 15B active. Designed for high-speed reasoning and agentic
workflows, it features hybrid attention and Multi-Token Prediction (MTP) to reduce
inference cost.

### Prerequisites

- Hardware: 4x H200 (TP4) or equivalent aggregate VRAM (~320 GB with FP8)
- vLLM >= 0.20.0

#### Install vLLM (NVIDIA)

```bash
uv venv
source .venv/bin/activate
uv pip install vllm --torch-backend auto
```

#### Install vLLM (AMD ROCm MI300X/MI325X/MI355X)

```bash
uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/
```

### Launch commands

Basic TP4:

```bash
vllm serve XiaomiMiMo/MiMo-V2-Flash \
  --host 0.0.0.0 --port 9001 --seed 1024 \
  --served-model-name mimo_v2_flash \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.9 \
  --generation-config vllm
```

With tool calling + reasoning:

```bash
vllm serve XiaomiMiMo/MiMo-V2-Flash \
  --tensor-parallel-size 4 --trust-remote-code --gpu-memory-utilization 0.9 \
  --tool-call-parser mimo \
  --reasoning-parser qwen3 \
  --generation-config vllm \
  --served-model-name mimo_v2_flash
```

DP + TP + EP:

```bash
vllm serve XiaomiMiMo/MiMo-V2-Flash \
  --data-parallel-size 2 \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --trust-remote-code \
  --gpu-memory-utilization 0.9 \
  --generation-config vllm \
  --served-model-name mimo_v2_flash
```

AMD:

```bash
export VLLM_ROCM_USE_AITER=0
vllm serve XiaomiMiMo/MiMo-V2-Flash --tensor-parallel-size 4 \
  --trust-remote-code --gpu-memory-utilization 0.9 --generation-config vllm
```

Tunable flags:
- `--max-model-len=65536` works well; max is 128K.
- `--max-num-batched-tokens=32768` for prompt-heavy; 16K/8K for lower latency.
- `--gpu-memory-utilization=0.95` to maximize KV cache.

### Client Usage

```bash
curl -X POST http://localhost:9001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mimo_v2_flash",
    "messages": [{"role": "user", "content": "Hello MiMo!"}],
    "chat_template_kwargs": {"enable_thinking": true}
  }'
```

Set `"enable_thinking": false` (or omit the kwargs) to disable thinking mode.

### Benchmarking

```bash
vllm bench serve \
  --model XiaomiMiMo/MiMo-V2-Flash \
  --dataset-name random --random-input-len 8000 --random-output-len 1000 \
  --request-rate 3 --num-prompts 1800 --ignore-eos
```

### Accuracy (GSM8K)

Reported 5-shot `exact_match`: flexible 0.9128, strict 0.9075.

### References

- [MiMo-V2-Flash on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash)