{ "hf_id": "zai-org/GLM-ASR-Nano-2512", "meta": { "title": "GLM-ASR-Nano-2512", "slug": "glm-asr-nano-2512", "provider": "GLM (Z-AI)", "description": "Open-source speech recognition model (~2B) with strong dialect support (Cantonese and others) and robust low-volume speech transcription", "date_added": "2026-01-26", "date_updated": "2026-04-17", "tasks": [ "multimodal" ], "performance_headline": "Outperforms Whisper V3 on multiple benchmarks at compact 1.5B active / 2B total size", "related_recipes": [] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve zai-org/GLM-ASR-Nano-2512 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "zai-org/GLM-ASR-Nano-2512", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest zai-org/GLM-ASR-Nano-2512 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "zai-org/GLM-ASR-Nano-2512", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/zai-org/GLM-ASR-Nano-2512/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/zai-org/GLM-ASR-Nano-2512/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/zai-org/GLM-ASR-Nano-2512/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/zai-org/GLM-ASR-Nano-2512/hw/h200.json", "h100": "/zai-org/GLM-ASR-Nano-2512/hw/h100.json", "b200": "/zai-org/GLM-ASR-Nano-2512/hw/b200.json", "gb200": "/zai-org/GLM-ASR-Nano-2512/hw/gb200.json", "b300": "/zai-org/GLM-ASR-Nano-2512/hw/b300.json", "gb300": "/zai-org/GLM-ASR-Nano-2512/hw/gb300.json", "mi300x": "/zai-org/GLM-ASR-Nano-2512/hw/mi300x.json", "mi325x": "/zai-org/GLM-ASR-Nano-2512/hw/mi325x.json", "mi350x": "/zai-org/GLM-ASR-Nano-2512/hw/mi350x.json", "mi355x": "/zai-org/GLM-ASR-Nano-2512/hw/mi355x.json" } }, "model": { "model_id": "zai-org/GLM-ASR-Nano-2512", "min_vllm_version": "0.14.1", "architecture": "dense", "parameter_count": "2.3B", "active_parameters": "1.5B", "context_length": 8192, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" }, "extras": [ { "note": "Audio extras required for ASR (requires vllm>=0.14.1)", "command": "uv pip install -U \"vllm[audio]\" --torch-backend auto" }, { "note": "Install transformers from source for GLM-ASR tokenizer support", "command": "uv pip install git+https://github.com/huggingface/transformers.git" } ] } }, "features": {}, "opt_in_features": [], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 11, "description": "Full precision BF16 — single-GPU deployment" } }, "kv_offload_support": { "offloading_cpu": "verified", "offloading_fs": "verified" }, "hardware_overrides": {}, "guide": "## Overview\n\nGLM-ASR-Nano-2512 is an open-source automatic speech recognition model with 1.5B\nactive parameters (2B total). It outperforms OpenAI Whisper V3 on multiple benchmarks\nwhile remaining compact enough for single-GPU deployment.\n\n### Key Capabilities\n\n- **Dialect support:** Beyond standard Mandarin and English, strong on Cantonese\n (粤语) and other Chinese dialects.\n- **Low-volume speech:** Specifically trained for \"whisper/quiet speech\" scenarios.\n- **SOTA accuracy:** Lowest average error rate (4.10) among comparable open-source\n models, strong on Wenet Meeting, Aishell-1, and similar Chinese benchmarks.\n\n## Prerequisites\n\n- **vLLM version:** >= 0.14.1 (with `[audio]` extras)\n- **Transformers:** install from source for latest\n\n### Install Dependencies\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install git+https://github.com/huggingface/transformers.git\nuv pip install -U \"vllm[audio]\" --torch-backend auto\n```\n\n## Launching the Server\n\n```bash\nvllm serve zai-org/GLM-ASR-Nano-2512\n```\n\n## Client Usage\n\n### OpenAI SDK (Audio URL)\n\n```python\nimport base64\nimport httpx\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\n\naudio_url = \"https://github.com/zai-org/GLM-ASR/raw/main/examples/example_en.wav\"\naudio_data = base64.b64encode(httpx.get(audio_url).content).decode(\"utf-8\")\n\nresponse = client.chat.completions.create(\n model=\"zai-org/GLM-ASR-Nano-2512\",\n messages=[{\n \"role\": \"user\",\n \"content\": [{\n \"type\": \"input_audio\",\n \"input_audio\": {\"data\": audio_data, \"format\": \"wav\"}\n }]\n }],\n max_tokens=500,\n)\nprint(response.choices[0].message.content)\n```\n\n### Transcribe Endpoint\n\n```python\nimport httpx\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\n\naudio_file = httpx.get(\"https://github.com/zai-org/GLM-ASR/raw/main/examples/example_en.wav\").content\n\nresponse = client.audio.transcriptions.create(\n model=\"zai-org/GLM-ASR-Nano-2512\",\n file=(\"audio.wav\", audio_file),\n)\nprint(response.text)\n```\n\n### cURL (Transcribe)\n\n```bash\ncurl http://localhost:8000/v1/audio/transcriptions \\\n -H \"Authorization: Bearer EMPTY\" \\\n -F \"model=zai-org/GLM-ASR-Nano-2512\" \\\n -F \"file=@your_audio.wav\"\n```\n\n### Local Audio File (chat API)\n\n```python\nimport base64\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\n\nwith open(\"your_audio.mp3\", \"rb\") as f:\n audio_data = base64.b64encode(f.read()).decode(\"utf-8\")\n\nresponse = client.chat.completions.create(\n model=\"zai-org/GLM-ASR-Nano-2512\",\n messages=[{\n \"role\": \"user\",\n \"content\": [{\"type\": \"input_audio\", \"input_audio\": {\"data\": audio_data, \"format\": \"mp3\"}}]\n }],\n max_tokens=500,\n)\nprint(response.choices[0].message.content)\n```\n\n## Troubleshooting\n\n- **Transformers version:** Requires `transformers >= 5.0.0` for best compatibility.\n- **Audio formats:** Supports wav, mp3, flac, and other common formats.\n\n## References\n\n- [Model card](https://huggingface.co/zai-org/GLM-ASR-Nano-2512)\n- [GitHub repo](https://github.com/zai-org/GLM-ASR)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/zai-org/GLM-ASR-Nano-2512.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/zai-org/GLM-ASR-Nano-2512.yaml", "overlays": [] } }