{ "hf_id": "Qwen/Qwen3-ASR-1.7B", "meta": { "title": "Qwen3-ASR-1.7B", "slug": "qwen3-asr-1.7b", "provider": "Qwen", "description": "Speech-to-text model supporting 11 languages, multiple accents, and singing voice with customizable text-context prompting.", "date_added": "2026-01-29", "date_updated": "2026-04-17", "tasks": [ "multimodal" ], "performance_headline": "Accurate multilingual ASR, including singing voice; single-GPU serving", "related_recipes": [] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve Qwen/Qwen3-ASR-1.7B \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "Qwen/Qwen3-ASR-1.7B", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest Qwen/Qwen3-ASR-1.7B \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "Qwen/Qwen3-ASR-1.7B", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy โ€” works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e ยท 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/Qwen/Qwen3-ASR-1.7B/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/Qwen/Qwen3-ASR-1.7B/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/Qwen/Qwen3-ASR-1.7B/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/Qwen/Qwen3-ASR-1.7B/hw/h200.json", "h100": "/Qwen/Qwen3-ASR-1.7B/hw/h100.json", "b200": "/Qwen/Qwen3-ASR-1.7B/hw/b200.json", "gb200": "/Qwen/Qwen3-ASR-1.7B/hw/gb200.json", "b300": "/Qwen/Qwen3-ASR-1.7B/hw/b300.json", "gb300": "/Qwen/Qwen3-ASR-1.7B/hw/gb300.json", "mi300x": "/Qwen/Qwen3-ASR-1.7B/hw/mi300x.json", "mi325x": "/Qwen/Qwen3-ASR-1.7B/hw/mi325x.json", "mi350x": "/Qwen/Qwen3-ASR-1.7B/hw/mi350x.json", "mi355x": "/Qwen/Qwen3-ASR-1.7B/hw/mi355x.json" } }, "model": { "model_id": "Qwen/Qwen3-ASR-1.7B", "min_vllm_version": "0.16.0", "architecture": "dense", "parameter_count": "2.3B", "active_parameters": "2.3B", "context_length": 65536, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" }, "extras": [ { "note": "Audio extras โ€” required for ASR input pre-processing (librosa, soundfile)", "command": "uv pip install -U \"vllm[audio]\"" } ] } }, "features": {}, "opt_in_features": [], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 4, "description": "Full precision BF16 โ€” fits on a single mid-range GPU" } }, "kv_offload_support": { "offloading_cpu": "verified", "offloading_fs": "verified" }, "hardware_overrides": { "amd": { "extra_args": [], "extra_env": { "VLLM_ROCM_USE_AITER": "1", "SAFETENSORS_FAST_GPU": "1" } } }, "guide": "## Overview\n\nQwen3-ASR is a speech-to-text model that achieves accurate and robust recognition across 11 languages and multiple accents. It supports prompting the model with text context in any format to produce customized ASR results and performs well on singing-voice recognition. This guide demonstrates how to deploy Qwen3-ASR efficiently with vLLM.\n\n## Prerequisites\n\nInstall vLLM with audio dependencies:\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --pre \\\n --extra-index-url https://wheels.vllm.ai/nightly/cu129 \\\n --extra-index-url https://download.pytorch.org/whl/cu129 \\\n --index-strategy unsafe-best-match\nuv pip install \"vllm[audio]\"\n```\n\n## Launching with vLLM\n\n```bash\nvllm serve Qwen/Qwen3-ASR-1.7B\n```\n\n## Client Usage\n\n### Chat Completions (OpenAI SDK)\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\n\nresponse = client.chat.completions.create(\n model=\"Qwen/Qwen3-ASR-1.7B\",\n messages=[{\n \"role\": \"user\",\n \"content\": [{\n \"type\": \"audio_url\",\n \"audio_url\": {\"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav\"},\n }],\n }],\n)\nprint(response.choices[0].message.content)\n```\n\n### Transcription API\n```python\nimport httpx\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\naudio_url = \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav\"\naudio_file = httpx.get(audio_url).content\n\ntranscription = client.audio.transcriptions.create(\n model=\"Qwen/Qwen3-ASR-1.7B\",\n file=audio_file,\n)\nprint(transcription.text)\n```\n\n### cURL\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"messages\": [\n {\"role\": \"user\", \"content\": [\n {\"type\": \"audio_url\", \"audio_url\": {\"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav\"}}\n ]}\n ]\n }'\n```\n\n### Offline Inference\n```python\nfrom vllm import LLM, SamplingParams\nfrom vllm.assets.audio import AudioAsset\n\nllm = LLM(model=\"Qwen/Qwen3-ASR-1.7B\")\naudio_asset = AudioAsset(\"winning_call\")\n\nconversation = [{\n \"role\": \"user\",\n \"content\": [{\"type\": \"audio_url\", \"audio_url\": {\"url\": audio_asset.url}}],\n}]\n\nsampling_params = SamplingParams(temperature=0.01, max_tokens=256)\noutputs = llm.chat(conversation, sampling_params=sampling_params)\nprint(outputs[0].outputs[0].text)\n```\n\n## Troubleshooting\n\n- Make sure `vllm[audio]` extras are installed or audio requests will fail.\n- Use the nightly wheel until Qwen3-ASR support lands in the stable release.\n\n## References\n\n- [Model card](https://huggingface.co/Qwen/Qwen3-ASR-1.7B)\n- [vLLM audio support](https://docs.vllm.ai/en/latest/features/multimodal_inputs.html)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/Qwen/Qwen3-ASR-1.7B.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/Qwen/Qwen3-ASR-1.7B.yaml", "overlays": [] } }