{ "hf_id": "mindlab-research/Macaron-V1-Coding-Venti", "meta": { "title": "Macaron-V1-Coding-Venti", "slug": "macaron-v1-coding-venti", "provider": "MindLab Research", "description": "Macaron-V1-Coding-Venti — MindLab's coding-specialist checkpoint: the Macaron-V1-Venti L2 Coding LoRA merged into the GLM-5.2 BF16 base. Same MoE architecture and launch as GLM-5.2 (~743B total, 39B active), no runtime adapter.", "date_added": "2026-07-28", "date_updated": "2026-07-23", "tasks": [ "text" ], "performance_headline": "Coding-specialist merged-LoRA on GLM-5.2 — tuned for SWE Verified, TerminalBench, DeepSWE, and agentic coding", "related_recipes": [ "zai-org/GLM-5.2" ], "hardware": { "b300": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "multi_node_tp", "variant": "default", "node_count": 2, "deploy_type": "multi_node", "env": { "GLOO_SOCKET_IFNAME": "$IFACE_NAME", "NCCL_SOCKET_IFNAME": "$IFACE_NAME" }, "docker_image": "vllm/vllm-openai:v0.28.0", "head_command": "vllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 0 \\\n --master-addr $HEAD_IP \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45", "worker_command": "vllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 1 \\\n --master-addr $HEAD_IP \\\n --headless \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45", "worker_commands": [ "vllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 1 \\\n --master-addr $HEAD_IP \\\n --headless \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45" ], "head_argv": [ "vllm", "serve", "mindlab-research/Macaron-V1-Coding-Venti", "--tensor-parallel-size", "16", "--nnodes", "2", "--node-rank", "0", "--master-addr", "$HEAD_IP", "--kv-cache-dtype", "fp8", "--tool-call-parser", "glm47", "--enable-auto-tool-choice", "--reasoning-parser", "glm45" ], "worker_argv": [ "vllm", "serve", "mindlab-research/Macaron-V1-Coding-Venti", "--tensor-parallel-size", "16", "--nnodes", "2", "--node-rank", "1", "--master-addr", "$HEAD_IP", "--headless", "--kv-cache-dtype", "fp8", "--tool-call-parser", "glm47", "--enable-auto-tool-choice", "--reasoning-parser", "glm45" ], "worker_argvs": [ [ "vllm", "serve", "mindlab-research/Macaron-V1-Coding-Venti", "--tensor-parallel-size", "16", "--nnodes", "2", "--node-rank", "1", "--master-addr", "$HEAD_IP", "--headless", "--kv-cache-dtype", "fp8", "--tool-call-parser", "glm47", "--enable-auto-tool-choice", "--reasoning-parser", "glm45" ] ], "head_docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n -e GLOO_SOCKET_IFNAME=$IFACE_NAME \\\n -e NCCL_SOCKET_IFNAME=$IFACE_NAME \\\n vllm/vllm-openai:v0.28.0 mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 0 \\\n --master-addr $HEAD_IP \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45", "worker_docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n -e GLOO_SOCKET_IFNAME=$IFACE_NAME \\\n -e NCCL_SOCKET_IFNAME=$IFACE_NAME \\\n vllm/vllm-openai:v0.28.0 mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 1 \\\n --master-addr $HEAD_IP \\\n --headless \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45", "worker_docker_commands": [ "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n -e GLOO_SOCKET_IFNAME=$IFACE_NAME \\\n -e NCCL_SOCKET_IFNAME=$IFACE_NAME \\\n vllm/vllm-openai:v0.28.0 mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 1 \\\n --master-addr $HEAD_IP \\\n --headless \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45" ], "head_docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "-e", "GLOO_SOCKET_IFNAME=$IFACE_NAME", "-e", "NCCL_SOCKET_IFNAME=$IFACE_NAME", "vllm/vllm-openai:v0.28.0", "mindlab-research/Macaron-V1-Coding-Venti", "--tensor-parallel-size", "16", "--nnodes", "2", "--node-rank", "0", "--master-addr", "$HEAD_IP", "--kv-cache-dtype", "fp8", "--tool-call-parser", "glm47", "--enable-auto-tool-choice", "--reasoning-parser", "glm45" ], "worker_docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "-e", "GLOO_SOCKET_IFNAME=$IFACE_NAME", "-e", "NCCL_SOCKET_IFNAME=$IFACE_NAME", "vllm/vllm-openai:v0.28.0", "mindlab-research/Macaron-V1-Coding-Venti", "--tensor-parallel-size", "16", "--nnodes", "2", "--node-rank", "1", "--master-addr", "$HEAD_IP", "--headless", "--kv-cache-dtype", "fp8", "--tool-call-parser", "glm47", "--enable-auto-tool-choice", "--reasoning-parser", "glm45" ], "worker_docker_argvs": [ [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "-e", "GLOO_SOCKET_IFNAME=$IFACE_NAME", "-e", "NCCL_SOCKET_IFNAME=$IFACE_NAME", "vllm/vllm-openai:v0.28.0", "mindlab-research/Macaron-V1-Coding-Venti", "--tensor-parallel-size", "16", "--nnodes", "2", "--node-rank", "1", "--master-addr", "$HEAD_IP", "--headless", "--kv-cache-dtype", "fp8", "--tool-call-parser", "glm47", "--enable-auto-tool-choice", "--reasoning-parser", "glm45" ] ], "strategy_spec": { "name": "multi_node_tp", "deploy_type": "multi_node", "display_name": "Multi-Node Tensor Parallel", "short_label": "Multi-node TP", "orientation": "latency", "description": "Multi-node TP. Splits the model across GPUs spanning multiple nodes. TP is set to total GPU count across all nodes. Head node serves requests, worker nodes run with --headless. Requires InfiniBand interconnect.\n", "hardware_match": { "min_gpus": 2, "multi_node": true }, "parallel_flag": "--tensor-parallel-size", "env": { "GLOO_SOCKET_IFNAME": "$IFACE_NAME", "NCCL_SOCKET_IFNAME": "$IFACE_NAME" } }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "single_node_tp": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/single_node_tp.json", "single_node_tep": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/single_node_tep.json", "multi_node_tp_pp": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/multi_node_tp_pp.json", "multi_node_tep": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/multi_node_tep.json", "multi_node_dep": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/multi_node_dep.json", "pd_cluster": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/pd_cluster.json", "kv_store_centralized_mooncake": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/mindlab-research/Macaron-V1-Coding-Venti/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/mindlab-research/Macaron-V1-Coding-Venti/hw/h200.json", "h100": "/mindlab-research/Macaron-V1-Coding-Venti/hw/h100.json", "b200": "/mindlab-research/Macaron-V1-Coding-Venti/hw/b200.json", "gb200": "/mindlab-research/Macaron-V1-Coding-Venti/hw/gb200.json", "b300": "/mindlab-research/Macaron-V1-Coding-Venti/hw/b300.json", "gb300": "/mindlab-research/Macaron-V1-Coding-Venti/hw/gb300.json", "mi300x": "/mindlab-research/Macaron-V1-Coding-Venti/hw/mi300x.json", "mi325x": "/mindlab-research/Macaron-V1-Coding-Venti/hw/mi325x.json", "mi350x": "/mindlab-research/Macaron-V1-Coding-Venti/hw/mi350x.json", "mi355x": "/mindlab-research/Macaron-V1-Coding-Venti/hw/mi355x.json" } }, "model": { "model_id": "mindlab-research/Macaron-V1-Coding-Venti", "min_vllm_version": "0.25.0", "docker_image": { "nvidia": "vllm/vllm-openai:v0.28.0", "amd": "vllm/vllm-openai-rocm:nightly" }, "architecture": "moe", "parameter_count": "743B", "active_parameters": "39B", "context_length": 1048576, "base_args": [ "--kv-cache-dtype", "fp8_e4m3" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:v0.28.0" } } }, "features": { "tool_calling": { "description": "GLM-4.7 tool call parser with automatic tool choice", "args": [ "--tool-call-parser", "glm47", "--enable-auto-tool-choice" ] }, "reasoning": { "description": "GLM-4.5 reasoning parser — thinking mode enabled by default on requests", "args": [ "--reasoning-parser", "glm45" ] }, "spec_decoding": { "description": "Multi-Token Prediction speculative decoding (up to 5 draft tokens)", "default_mode": "mtp", "modes": { "mtp": { "label": "MTP", "description": "Built-in Multi-Token Prediction inherited from the GLM-5.2 base, 5 draft tokens.", "variants": [ "default", "fp8" ], "args": [ "--speculative-config", "{\"method\":\"mtp\",\"num_speculative_tokens\":5}" ] }, "dspark": { "label": "DSpark", "description": "DSpark drafting with the externally trained GLM-5.2 7-token draft model; the served Macaron checkpoint is unchanged.", "variants": [ "default" ], "args": [ "--speculative-config", "{\"model\": \"RedHatAI/GLM-5.2-speculator.dspark\",\"num_speculative_tokens\": 7,\"method\": \"dspark\"}" ] } } } }, "opt_in_features": [ "spec_decoding" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 1786, "description": "Full-precision BF16 merged checkpoint — requires multi-node (≈1786 GB weights + KV; a single 8xH200/8xB200 node does not fit).", "hardware_overrides": { "amd": { "extra_args": [ "--linear-backend", "aiter" ] }, "mi355x": { "extra_args": [ "--max-model-len", "262144" ] } } }, "fp8": { "precision": "fp8", "label": "FP8 (on-the-fly)", "vram_minimum_gb": 893, "description": "Same BF16 checkpoint, quantized to FP8 at load time via --quantization fp8 — not a native FP8 weight set. Halves weight memory so it fits a single node; on a B300 node it leaves ~2.7× the GPU KV-cache space of BF16 (≈3.07M vs 1.12M). Slower startup; on GSM8K (5-shot, exact-match) it matches native BF16 within noise (0.9424 vs 0.9431, ±0.0064).", "extra_args": [ "--quantization", "fp8" ], "hardware_overrides": { "amd": { "extra_args": [ "--quantization", "fp8_per_channel" ] } } } }, "hardware_overrides": { "hopper": { "extra_args": [ "--kv-cache-dtype", "fp8" ] }, "amd": { "extra_args": [ "--moe-backend", "aiter" ], "extra_env": { "VLLM_ROCM_USE_AITER": "1", "VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS": "1" } } }, "guide": "## Overview\n\n**Macaron-V1-Coding-Venti** is MindLab Research's coding-specialist checkpoint in the\nMacaron-V1 family. It is produced by **merging the Macaron-V1-Venti L2 Coding LoRA\nspecialist into the GLM-5.2 BF16 base**, yielding a single merged checkpoint — no PEFT\nadapter or Mixture-of-LoRA (MoL) router is needed at runtime. It targets code\nunderstanding, repository-level software engineering, terminal use, and coding-agent\nworkflows, and reports strong coding/terminal slices (SWE Verified 85.6, TerminalBench 2.1\n87.6, DeepSWE 58.4) on the Macaron evaluation suite.\n\nBecause it is a weight-level merge of a LoRA onto GLM-5.2, the runtime architecture is\n**identical to GLM-5.2** — same `GlmMoeDsaForCausalLM` MoE (~743B total / 39B active), same\n1M native context, same MTP indexer. **Launch it with the same serving stack as [GLM-5.2 recipe](/zai-org/GLM-5.2).**\n\nThis model ships **BF16 only** (no native FP8 / NVFP4 checkpoint). At full BF16 precision\n(≈1786 GB) it needs multi-node; the **FP8 (on-the-fly)** variant loads the same weights as\nFP8 (`--quantization fp8`, ≈893 GB) to fit a single 8×H200 / 8×B200 node — or to free VRAM\non larger trays.\n\n## Prerequisites\n\n- **vLLM 0.25.0** (stable) — the `GlmMoeDsaForCausalLM` handler is the same one GLM-5.2 uses.\n- **GPU:** at full **BF16** precision (≈1786 GB) a single 8×H200 / 8×B200 node does not fit —\n run **multi-node** (e.g. 2× 8-GPU nodes, tensor-parallel 16), or pick the **FP8 (on-the-fly)**\n variant (≈893 GB, `--quantization fp8`; `fp8_per_channel` on AMD) to run single-node. The native FP8/NVFP4 GLM-5.2\n checkpoints are a calibrated-quant alternative.\n\n## Installation\n\n### Docker\n\nThe BF16 variant needs two nodes (the FP8 (on-the-fly) variant fits one — see Launching\nbelow). Run one container per node (each `--gpus all`, 8 local GPUs); both pass\n`--tensor-parallel-size 16 --nnodes 2`, varying `--node-rank`. This is the head node —\nnode 1 repeats with `--node-rank 1 --headless`.\n\n```bash\ndocker run --gpus all \\\n -p 8000:8000 \\\n --ipc=host \\\n --network host \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:v0.28.0 mindlab-research/Macaron-V1-Coding-Venti \\\n --tensor-parallel-size 16 \\\n --nnodes 2 \\\n --node-rank 0 \\\n --master-addr $HEAD_IP \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --reasoning-parser glm45 \\\n --enable-auto-tool-choice \\\n --served-model-name macaron-v1-coding-venti\n```\n\n### From source\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install \"vllm==0.25.0\" --torch-backend=auto\nuv pip install \"transformers>=5.9.0\"\n```\n\n## Launching the server\n\nPick the **Variant** that matches your hardware: **BF16** for full precision on multi-node,\nor **FP8 (on-the-fly)** to fit a single node (or free VRAM on a larger tray). The builder\nrenders the exact flags per your Variant / Strategy / Nodes picks.\n\n### BF16 — 2× 8xH200 (multi-node TP)\n\nFull-precision BF16 (≈1786 GB) needs multi-node. The command below is a 2-node\ntensor-parallel example (16 GPUs). Run the head command on node 0 and the worker on node 1,\npointing `--master-addr` at the head's IP. Pick the **Multi-node TP** strategy in the\nbuilder to render both tabs.\n\n```bash\n# Node 0 (head)\nvllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --kv-cache-dtype fp8 \\\n --tensor-parallel-size 16 \\\n --tool-call-parser glm47 \\\n --reasoning-parser glm45 \\\n --enable-auto-tool-choice \\\n --served-model-name macaron-v1-coding-venti\n\n# Node 1\nvllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --kv-cache-dtype fp8 \\\n --tensor-parallel-size 16 \\\n --tool-call-parser glm47 \\\n --reasoning-parser glm45 \\\n --enable-auto-tool-choice \\\n --served-model-name macaron-v1-coding-venti\n```\n\n> The builder emits the exact multi-node multiprocessing args (`--nnodes`,\n> `--node-rank`, `--master-addr` for TP) when you select **Multi-node TP**.\n\n### FP8 (on-the-fly) on a single node\n\nPick the **FP8 (on-the-fly)** variant to load the same BF16 checkpoint as FP8 via\n`--quantization fp8` (runtime quantization — not a native FP8 weight set). At ≈893 GB it\nfits a single 8×H200 / 8×B200 node, and on larger trays (e.g. B300) it frees VRAM for\nlonger context or more concurrent sequences. Startup is slower while the quantization pass\nruns at load; the accuracy / KV-cache trade-off is small (see the B300 numbers below).\n\n```bash\nvllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --quantization fp8 \\\n --tensor-parallel-size 8 \\\n --kv-cache-dtype fp8 \\\n --tool-call-parser glm47 \\\n --reasoning-parser glm45 \\\n --enable-auto-tool-choice \\\n --served-model-name macaron-v1-coding-venti\n```\n\n- **`--max-model-len`** — raise toward the 1M window when KV-cache headroom allows; lower on OOM.\n- **`--max-num-seqs 32`** — the main knob for fitting long context under concurrency.\n- **`--gpu-memory-utilization 0.80`** — leaves ROCm runtime headroom for MTP graph capture.\n\nOn a **B300** node, on-the-fly FP8 trades a negligible accuracy drop for a large KV-cache\nheadroom gain:\n\n| Variant | GSM8K 5-shot (exact-match) | B300 GPU KV-cache size |\n|---|---:|---:|\n| BF16 | 0.9431 (±0.0064) | ≈1.12M |\n| FP8 (on-the-fly) | 0.9424 (±0.0064) | ≈3.07M |\n\nThe GSM8K scores sit within stderr of each other, while FP8 leaves ~2.7× the KV-cache space\n— so on B300 (where BF16 already fits) FP8 buys far more concurrency / context at almost no\naccuracy cost.\n\n### Speculative decoding (MTP)\n\nThe merged weights inherit GLM-5.2's built-in Multi-Token Prediction head, so MTP works\nexactly as on GLM-5.2 — on both the BF16 and FP8 (on-the-fly) variants. Toggle\n**spec_decoding → MTP** in the builder:\n\n```bash\nvllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --kv-cache-dtype fp8 \\\n --tensor-parallel-size 16 \\\n --speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":5}' \\\n --tool-call-parser glm47 \\\n --reasoning-parser glm45 \\\n --enable-auto-tool-choice \\\n --served-model-name macaron-v1-coding-venti\n```\n\n### AMD MI300X / MI355X\n\nUse the AITER backends under ROCm, same as GLM-5.2. On AMD the FP8 (on-the-fly) variant\nuses `--quantization fp8_per_channel`.\n\nFP8 (on-the-fly):\n\n```bash\nVLLM_ROCM_USE_AITER=1 \\\nVLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 \\\nvllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --kv-cache-dtype fp8_e4m3 \\\n --tensor-parallel-size 8 \\\n --quantization fp8_per_channel \\\n --moe-backend aiter \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45\n```\n\nBF16:\n\n```bash\nVLLM_ROCM_USE_AITER=1 \\\nVLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 \\\nvllm serve mindlab-research/Macaron-V1-Coding-Venti \\\n --kv-cache-dtype fp8_e4m3 \\\n --tensor-parallel-size 8 \\\n --max-model-len 262144 \\\n --linear-backend aiter \\\n --moe-backend aiter \\\n --tool-call-parser glm47 \\\n --enable-auto-tool-choice \\\n --reasoning-parser glm45\n```\n\nOn 8×MI355X, GSM8K 5-shot (strict-match) is 0.9469 for both BF16 and FP8 (on-the-fly).\n\n## Reasoning modes\n\nThinking is **on by default** and reuses the GLM/DeepSeek-V4 `reasoning_effort` mechanism,\nidentical to GLM-5.2. The chat template resolves effort to **`max` unless\n`reasoning_effort` is explicitly `\"high\"`**.\n\n| Mode | How to request | Behavior |\n|------|----------------|----------|\n| **Think Max** (default) | omit `reasoning_effort`, or set `\"max\"` | Deepest reasoning — hard math, multi-step planning, agentic tasks. |\n| **Think High** | `\"reasoning_effort\": \"high\"` | Balanced depth and latency. |\n| **Non-think** | `chat_template_kwargs.enable_thinking: false` | Fast, no chain-of-thought. |\n\n## Client usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\nmsgs = [{\"role\": \"user\", \"content\": \"Refactor this function to handle empty input.\"}]\n\n# Think Max (default) — just omit reasoning_effort\nclient.chat.completions.create(model=\"macaron-v1-coding-venti\", messages=msgs, max_tokens=4096)\n\n# Think High\nclient.chat.completions.create(\n model=\"macaron-v1-coding-venti\",\n messages=msgs,\n max_tokens=4096,\n extra_body={\"chat_template_kwargs\": {\"reasoning_effort\": \"high\"}},\n)\n\n# Non-think\nclient.chat.completions.create(\n model=\"macaron-v1-coding-venti\",\n messages=msgs,\n max_tokens=4096,\n extra_body={\"chat_template_kwargs\": {\"enable_thinking\": False}},\n)\n```\n\n## Benchmarking\n\nAdd `--no-enable-prefix-caching` to the server command for a clean measurement.\n\n```bash\nvllm bench serve \\\n --model mindlab-research/Macaron-V1-Coding-Venti \\\n --dataset-name random \\\n --random-input 8000 \\\n --random-output 1024 \\\n --request-rate 10 \\\n --num-prompts 32 \\\n --ignore-eos\n```\n\n> **Note:** pure throughput benchmarks tend to under-report real speed, because MTP's\n> acceptance rate is usually low in synthetic runs.\n\n## References\n\n- [Model card](https://huggingface.co/mindlab-research/Macaron-V1-Coding-Venti)\n- [Base model: GLM-5.2 recipe](/zai-org/GLM-5.2)\n- [Introducing Macaron-V1 (blog)](https://macaron.im/mindlab/research/introducing-macaron-v1)\n- [Macaron Artifacts (GitHub)](https://github.com/MindLab-Research/macaron-artifacts)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/mindlab-research/Macaron-V1-Coding-Venti.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/mindlab-research/Macaron-V1-Coding-Venti.yaml", "overlays": [] } }