{
  "hf_id": "openbmb/MiniCPM5-1B",
  "meta": {
    "title": "MiniCPM5-1B",
    "slug": "minicpm5-1b",
    "provider": "MiniCPM (OpenBMB)",
    "description": "MiniCPM5-1B — dense 1B on-device LLM with hybrid Think/No-Think reasoning, native 128K context, and strong agentic tool use, built on the standard Llama architecture",
    "date_added": "2026-06-02",
    "date_updated": "2026-06-02",
    "tasks": [
      "text"
    ],
    "performance_headline": "1B-class open-source SOTA on tool use, code, and reasoning",
    "related_recipes": [
      "openbmb/MiniCPM-V-4.6"
    ]
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:latest",
    "command": "vllm serve openbmb/MiniCPM5-1B \\\n  --tensor-parallel-size 1",
    "argv": [
      "vllm",
      "serve",
      "openbmb/MiniCPM5-1B",
      "--tensor-parallel-size",
      "1"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:latest openbmb/MiniCPM5-1B \\\n  --tensor-parallel-size 1",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:latest",
      "openbmb/MiniCPM5-1B",
      "--tensor-parallel-size",
      "1"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "multi_node_tp": "/openbmb/MiniCPM5-1B/strategies/multi_node_tp.json",
      "kv_store_centralized_mooncake": "/openbmb/MiniCPM5-1B/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/openbmb/MiniCPM5-1B/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/openbmb/MiniCPM5-1B/hw/h200.json",
      "h100": "/openbmb/MiniCPM5-1B/hw/h100.json",
      "b200": "/openbmb/MiniCPM5-1B/hw/b200.json",
      "gb200": "/openbmb/MiniCPM5-1B/hw/gb200.json",
      "b300": "/openbmb/MiniCPM5-1B/hw/b300.json",
      "gb300": "/openbmb/MiniCPM5-1B/hw/gb300.json",
      "mi300x": "/openbmb/MiniCPM5-1B/hw/mi300x.json",
      "mi325x": "/openbmb/MiniCPM5-1B/hw/mi325x.json",
      "mi350x": "/openbmb/MiniCPM5-1B/hw/mi350x.json",
      "mi355x": "/openbmb/MiniCPM5-1B/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "openbmb/MiniCPM5-1B",
    "min_vllm_version": "0.21.0",
    "architecture": "dense",
    "parameter_count": "1.1B",
    "active_parameters": "1.1B",
    "context_length": 131072,
    "base_args": [],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "reasoning": {
      "description": "Enable hybrid deep-thinking mode — the chat template wraps a <think> block before the answer. Recommended sampling shifts to temperature=0.9, top_p=0.95. Leave off for fast No-Think responses (temperature=0.7).",
      "args": [
        "--default-chat-template-kwargs",
        "{\"enable_thinking\": true}"
      ]
    }
  },
  "opt_in_features": [
    "reasoning"
  ],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 3,
      "description": "BF16 weights (~1.1B params) — tiny footprint, runs on a single consumer GPU"
    }
  },
  "kv_offload_support": {
    "offloading_cpu": "verified",
    "offloading_fs": "verified"
  },
  "guide": "## Overview\n\n**MiniCPM5-1B** is the first checkpoint in OpenBMB's MiniCPM5 series — a dense\n1B model built for on-device and resource-constrained deployment, reaching\n1B-class open-source SOTA on agentic tool use, code generation, and difficult\nreasoning. It uses the **standard `LlamaForCausalLM` architecture**, so vLLM\nloads it natively with no custom kernels or model-code fork.\n\nIts headline feature is **hybrid reasoning**: a single checkpoint serves as\nboth a fast assistant (No-Think) and a deliberate reasoner (Think), toggled by\nthe chat template's `enable_thinking` flag.\n\n## Prerequisites\n\n- **vLLM ≥ 0.21.0** — MiniCPM5-1B is supported natively as of the v0.21.0\n  release. (For CUDA 12.x driver hosts, the cookbook suggests `vllm==0.10.1.1`\n  as a fallback.)\n\n## Launch command\n\nUse the command builder above. The baseline is simply:\n\n```bash\nvllm serve openbmb/MiniCPM5-1B --port 8000\n```\n\nAt ~1.1B params the model fits on a single GPU (TP=1). It supports the full\nnative 128K context; drop `--max-model-len` to `8192` / `32768` to free KV\ncache on small GPUs, and set `--enforce-eager` if CUDA graphs OOM on a tiny\nVRAM budget.\n\n## Reasoning modes\n\nToggle the **Reasoning** feature to serve with deep-thinking on by default\n(`--default-chat-template-kwargs '{\"enable_thinking\": true}'`). You can also\nflip it per request via `chat_template_kwargs`:\n\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"openbmb/MiniCPM5-1B\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Explain GQA in one sentence.\"}],\n    \"temperature\": 0.9, \"top_p\": 0.95, \"max_tokens\": 1024,\n    \"chat_template_kwargs\": {\"enable_thinking\": true}\n  }'\n```\n\n| Mode | `enable_thinking` | `temperature` | `top_p` |\n| --- | --- | --- | --- |\n| Think | `true` | 0.9 | 0.95 |\n| No-Think | `false` | 0.7 | 0.95 |\n\n## Tool calling\n\nMiniCPM5-1B emits **XML-style** tool calls. The vLLM-side `minicpm5` parser\n([PR #43175](https://github.com/vllm-project/vllm/pull/43175)) merged to `main`\non 2026-05-27 but is **not in v0.21.0 or v0.22.0** — those releases were cut\nbefore the merge. Until a release bakes it in (v0.23+), load it as a plugin\nfrom the [MiniCPM repo](https://github.com/OpenBMB/MiniCPM):\n\n```bash\nvllm serve openbmb/MiniCPM5-1B --port 8000 \\\n  --enable-auto-tool-choice \\\n  --tool-parser-plugin /path/to/MiniCPM/tool_parsers/minicpm5xml_tool_parser.py \\\n  --tool-call-parser minicpm5\n```\n\nSGLang ships the `minicpm5` parser built-in and is the author-recommended\nbackend for tool calling.\n\n## References\n\n- [Model card](https://huggingface.co/openbmb/MiniCPM5-1B)\n- [vLLM deployment cookbook](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm.md)\n- [MiniCPM Tech Report](https://arxiv.org/pdf/2506.07900)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "389ffebe555705c85337025d5df108d0384d365e",
    "path": "recipes/models/openbmb/MiniCPM5-1B.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/openbmb/MiniCPM5-1B.yaml",
    "overlays": []
  }
}