{ "hf_id": "openbmb/MiniCPM-V-4.6", "meta": { "title": "MiniCPM-V 4.6", "slug": "minicpm-v-4-6", "provider": "MiniCPM (OpenBMB)", "description": "MiniCPM-V 4.6 (1.3B) — pocket-sized multimodal LLM for ultra-efficient single-image, multi-image, and video understanding, built on SigLIP2-400M + a Qwen3.5-0.8B hybrid-attention backbone", "date_added": "2026-06-02", "date_updated": "2026-06-02", "tasks": [ "multimodal" ], "performance_headline": "~1.5× token throughput vs Qwen3.5-0.8B with mixed 4×/16× visual token compression", "related_recipes": [ "openbmb/MiniCPM5-1B" ] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve openbmb/MiniCPM-V-4.6 \\\n --trust-remote-code \\\n --tensor-parallel-size 1 \\\n --enable-auto-tool-choice \\\n --tool-call-parser qwen3_coder", "argv": [ "vllm", "serve", "openbmb/MiniCPM-V-4.6", "--trust-remote-code", "--tensor-parallel-size", "1", "--enable-auto-tool-choice", "--tool-call-parser", "qwen3_coder" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest openbmb/MiniCPM-V-4.6 \\\n --trust-remote-code \\\n --tensor-parallel-size 1 \\\n --enable-auto-tool-choice \\\n --tool-call-parser qwen3_coder", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "openbmb/MiniCPM-V-4.6", "--trust-remote-code", "--tensor-parallel-size", "1", "--enable-auto-tool-choice", "--tool-call-parser", "qwen3_coder" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/openbmb/MiniCPM-V-4.6/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/openbmb/MiniCPM-V-4.6/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/openbmb/MiniCPM-V-4.6/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/openbmb/MiniCPM-V-4.6/hw/h200.json", "h100": "/openbmb/MiniCPM-V-4.6/hw/h100.json", "b200": "/openbmb/MiniCPM-V-4.6/hw/b200.json", "gb200": "/openbmb/MiniCPM-V-4.6/hw/gb200.json", "b300": "/openbmb/MiniCPM-V-4.6/hw/b300.json", "gb300": "/openbmb/MiniCPM-V-4.6/hw/gb300.json", "mi300x": "/openbmb/MiniCPM-V-4.6/hw/mi300x.json", "mi325x": "/openbmb/MiniCPM-V-4.6/hw/mi325x.json", "mi350x": "/openbmb/MiniCPM-V-4.6/hw/mi350x.json", "mi355x": "/openbmb/MiniCPM-V-4.6/hw/mi355x.json" } }, "model": { "model_id": "openbmb/MiniCPM-V-4.6", "min_vllm_version": "0.22.0", "architecture": "dense", "parameter_count": "1.3B", "active_parameters": "1.3B", "context_length": 262144, "base_args": [ "--trust-remote-code" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" }, "extras": [ { "note": "MiniCPM-V 4.6 ships as a standalone architecture (MiniCPMV4_6ForConditionalGeneration) only in transformers >= 5.7.0", "command": "uv pip install -U \"transformers>=5.7.0\"" }, { "note": "Video understanding support (decoders for the video input path)", "command": "uv pip install -U \"vllm[video]\"", "optional": true } ] } }, "features": { "tool_calling": { "description": "Function/tool calling via the qwen3_coder parser (the v4.6 chat template emits Qwen3-Coder-style blocks)", "args": [ "--enable-auto-tool-choice", "--tool-call-parser", "qwen3_coder" ] } }, "opt_in_features": [], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 4, "description": "BF16 weights (~1.3B params) — tiny footprint, runs comfortably on a single consumer GPU" } }, "guide": "## Overview\n\n**MiniCPM-V 4.6** is OpenBMB's most edge-friendly multimodal model to date.\nIt pairs a **SigLIP2-400M** vision encoder with a **Qwen3.5-0.8B** hybrid\nlinear/full-attention language backbone (~1.3B params total). It handles\nsingle-image, multi-image, and video understanding, and introduces **mixed\n4×/16× visual token compression** — switch the `downsample_mode` per request\nto trade accuracy for speed.\n\n> MiniCPM-V 4.6 ships as **two independent checkpoints** — this recipe covers\n> the Instruct model (`openbmb/MiniCPM-V-4.6`). For chain-of-thought output,\n> use the separate `openbmb/MiniCPM-V-4.6-Thinking` checkpoint instead (unlike\n> v4.5, the mode is no longer toggled at runtime).\n\n## Prerequisites\n\n- **vLLM ≥ 0.22.0** — the `MiniCPMV4_6ForConditionalGeneration` architecture\n landed via [PR #43213](https://github.com/vllm-project/vllm/pull/43213)\n (merged 2026-05-22) and first shipped in the v0.22.0 release. No fork is\n required.\n- **transformers ≥ 5.7.0** — the model is merged as a standalone architecture\n only in this version.\n- For video inputs, install the `vllm[video]` extra.\n\n## Launch command\n\nUse the command builder above. The baseline is simply:\n\n```bash\nvllm serve openbmb/MiniCPM-V-4.6 --trust-remote-code --port 8000\n```\n\nEnable the **Tool calling** feature to add `--enable-auto-tool-choice\n--tool-call-parser qwen3_coder`. The model emits a Qwen3-Coder-style\n`` block inside the message content.\n\nAt 1.3B params the model fits on a single GPU (TP=1). v4.6 supports context up\nto 256K, but start with a smaller `--max-model-len` (e.g. `8192`) and raise it\nto fit your GPU memory and workload.\n\n## Client usage\n\n**Image:**\n\n```bash\ncurl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{\n \"model\": \"openbmb/MiniCPM-V-4.6\",\n \"messages\": [{\"role\": \"user\", \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://huggingface.co/datasets/openbmb/DemoCase/resolve/main/refract.png\"}},\n {\"type\": \"text\", \"text\": \"What causes this phenomenon?\"}\n ]}]\n}'\n```\n\n> **Stop tokens:** v4.6 uses the Qwen3.5 backbone with a new vocab. If you see\n> runaway generations, pass `\"stop_token_ids\": [248044, 248046]` in your\n> request body.\n\n## References\n\n- [Model card](https://huggingface.co/openbmb/MiniCPM-V-4.6)\n- [Thinking checkpoint](https://huggingface.co/openbmb/MiniCPM-V-4.6-Thinking)\n- [vLLM deployment guide (Cookbook)](https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/vllm/minicpm-v4_6_vllm.md)\n- [vLLM support PR #43213](https://github.com/vllm-project/vllm/pull/43213)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/openbmb/MiniCPM-V-4.6.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/openbmb/MiniCPM-V-4.6.yaml", "overlays": [] } }