{ "hf_id": "vllm-sr/d3-mini", "meta": { "title": "d3-mini", "slug": "d3-mini", "provider": "vLLM Semantic Router", "description": "Decision 3.0's 4B decision model: answers typed questions (choice, yes / no, score) about text, JSON, images and videos with a probability for every option, without generating text.", "date_added": "2026-10-15", "date_updated": "2026-10-15", "tasks": [ "embedding", "multimodal" ], "performance_headline": "MI300X: 415 of 416 decisions match the checkpoint runtime (one near-tie); 2.8x its server's requests per second at 64 clients", "related_recipes": [ "vllm-sr/d3", "vllm-sr/d3-flash", "vllm-sr/d3-nano", "vllm-sr/d3-lite", "vllm-sr/d3-edge" ], "default_hardware": "mi300x", "hardware": { "mi300x": "verified" } }, "recommended_command": { "hardware": "mi300x", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai-rocm:nightly", "command": "vllm serve vllm-sr/d3-mini \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "vllm-sr/d3-mini", "--tensor-parallel-size", "1" ], "docker_command": "docker run --device=/dev/kfd --device=/dev/dri \\\n --security-opt seccomp=unconfined --group-add video \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai-rocm:nightly vllm-sr/d3-mini \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--device=/dev/kfd", "--device=/dev/dri", "--security-opt", "seccomp=unconfined", "--group-add", "video", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai-rocm:nightly", "vllm-sr/d3-mini", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "AMD", "generation": "amd", "compute_arch": "gfx942", "display_name": "MI300X", "description": "AMD Instinct MI300X 192 GB HBM3 · 8-GPU OAM node (CDNA 3)", "gpu_count": 8, "vram_gb": 1536, "multi_node": false }, "alternatives": { "kv_store_centralized_mooncake": "/vllm-sr/d3-mini/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/vllm-sr/d3-mini/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "mi300x": "/vllm-sr/d3-mini/hw/mi300x.json", "h100": "/vllm-sr/d3-mini/hw/h100.json", "h200": "/vllm-sr/d3-mini/hw/h200.json", "b200": "/vllm-sr/d3-mini/hw/b200.json", "gb200": "/vllm-sr/d3-mini/hw/gb200.json", "b300": "/vllm-sr/d3-mini/hw/b300.json", "gb300": "/vllm-sr/d3-mini/hw/gb300.json", "mi325x": "/vllm-sr/d3-mini/hw/mi325x.json", "mi350x": "/vllm-sr/d3-mini/hw/mi350x.json", "mi355x": "/vllm-sr/d3-mini/hw/mi355x.json" } }, "model": { "model_id": "vllm-sr/d3-mini", "min_vllm_version": "nightly", "nightly_required": true, "architecture": "dense", "parameter_count": "4.54B", "active_parameters": "4.54B", "context_length": 262144, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv --python 3.12 --seed\nsource .venv/bin/activate\n\n# NVIDIA GPUs (CUDA)\nuv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly\n\n# AMD GPUs (ROCm 7.2; glibc 2.39 or newer, as on Ubuntu 24.04)\nsudo apt install -y libopenmpi3t64 libdw1t64\nexport VLLM_ROCM_VARIANT=$(curl -s https://wheels.vllm.ai/rocm/nightly | grep -oP 'rocm\\d+' | head -1 | sed 's/%2B/+/g')\nuv pip install --pre vllm --extra-index-url https://wheels.vllm.ai/rocm/nightly/${VLLM_ROCM_VARIANT} --index-strategy unsafe-best-match\n", "note": "Decision 3.0 is on vLLM main since 2026-10-14 (vllm-project/vllm#61565) and not in a release yet: install a nightly." } } }, "features": {}, "opt_in_features": [], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 11, "description": "BF16 weights (9.1 GB) and an FP32 answer-code readout" } }, "hardware_overrides": {}, "guide": "## Overview\n\n[vllm-sr/d3-mini](https://huggingface.co/vllm-sr/d3-mini) is the 4B model of Decision 3.0, the decision models of\n[vLLM Semantic Router](https://github.com/vllm-project/semantic-router). Give it a state (text or JSON,\noptionally with images and videos) and typed questions: pick one of several options (`choice`), yes or no\n(`noul`), or a level on a scale (`score`). It returns a probability for every option, without generating text.\n\nIt is a backbone of the Qwen3.5 architecture (4.54B parameters, including the 0.33B vision\nencoder), trained from `Qwen/Qwen3.5-4B`, with an answer-code readout. vLLM detects the checkpoint's `decision_config.json` and serves it\nas a classification model (`D3ForSequenceClassification`) on its System One endpoint, `/v1/systemone`: one\nforward pass per question, whose readout scores that question's options at the last token. The full attention\nlayers read the prompt in both directions, so chunked prefill and prefix caching are off.\n\nThe Decision 3.0 models, all served the same way:\n\n- [`vllm-sr/d3`](https://huggingface.co/vllm-sr/d3) (27B)\n- [`vllm-sr/d3-flash`](https://huggingface.co/vllm-sr/d3-flash) (9B)\n- [`vllm-sr/d3-mini`](https://huggingface.co/vllm-sr/d3-mini) (4B) — this recipe\n- [`vllm-sr/d3-nano`](https://huggingface.co/vllm-sr/d3-nano) (2B)\n- [`vllm-sr/d3-lite`](https://huggingface.co/vllm-sr/d3-lite) (0.8B)\n- [`vllm-sr/d3-edge`](https://huggingface.co/vllm-sr/d3-edge) (0.6B)\n\n## Prerequisites\n\n- **Hardware**: one GPU. The BF16 weights take 9.1 GB; vLLM reserves 92 % of the GPU's memory by default.\n- **vLLM**: a nightly build. Decision 3.0 support ([vllm-project/vllm#61565](https://github.com/vllm-project/vllm/pull/61565))\n is on vLLM main since 2026-10-14 and not in a release yet (the latest, v0.31.0, does not include it).\n Every nightly built since 2026-10-15 includes it.\n- **ROCm**: Python 3.12, glibc 2.39 or newer (for example Ubuntu 24.04), and the system libraries\n `libopenmpi3t64` and `libdw1t64`, which the ROCm wheels load at import.\n\n```bash\nuv venv --python 3.12 --seed\nsource .venv/bin/activate\n\n# NVIDIA GPUs (CUDA)\nuv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly\n\n# AMD GPUs (ROCm 7.2; glibc 2.39 or newer, as on Ubuntu 24.04)\nsudo apt install -y libopenmpi3t64 libdw1t64\nexport VLLM_ROCM_VARIANT=$(curl -s https://wheels.vllm.ai/rocm/nightly | grep -oP 'rocm\\d+' | head -1 | sed 's/%2B/+/g')\nuv pip install --pre vllm --extra-index-url https://wheels.vllm.ai/rocm/nightly/${VLLM_ROCM_VARIANT} --index-strategy unsafe-best-match\n```\n\n## Launch\n\n```bash\nvllm serve vllm-sr/d3-mini\n```\n\nNo flags are needed; the first start downloads the model.\n\n## Client usage\n\nThe command builder's cURL and Bench buttons send chat completions, which a decision model does not\nserve. Ask questions on `/v1/systemone` instead:\n\n```bash\ncurl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{\n \"model\": \"vllm-sr/d3-mini\",\n \"state\": \"Write a Python function that merges two sorted lists.\",\n \"questions\": {\n \"kind\": {\"type\": \"choice\", \"instructions\": \"What kind of work is this?\",\n \"criteria\": {\"code\": \"Writing or fixing code\", \"math\": \"Mathematics\", \"chat\": \"Anything else\"}},\n \"reasoning\": {\"type\": \"noul\", \"instructions\": \"Does answering this need multi-step reasoning?\"}\n }\n}'\n```\n\nEach answer carries the pick, the probability of every option, and `confidence`, the chosen option's\nprobability; a `noul` answer's value is the probability of yes. The probabilities are the checkpoint\nruntime's (`model.system_one()`). Images and videos go in `images` and `videos` as http(s) or base64 data\nURLs; vLLM places them before the text and reads them with the runtime's budgets (images up to 1.6\nmegapixels; videos at 2 fps, at most 32 frames):\n\n```bash\ncurl -s localhost:8000/v1/systemone -H 'content-type: application/json' --data-binary @- <