{ "hf_id": "Qwen/Qwen3-VL-30B-A3B-Instruct", "meta": { "title": "Qwen3-VL-30B-A3B-Instruct", "slug": "qwen3-vl-30b-a3b-instruct", "provider": "Qwen", "description": "Qwen3-VL MoE vision-language model with 30B total / 3B active parameters, supporting image, video, and text workloads.", "date_added": "2026-08-17", "date_updated": "2026-09-12", "tasks": [ "multimodal", "text" ], "performance_headline": "Validated on Intel Xeon 6 CPU with BF16 serving", "related_recipes": [ "Qwen/Qwen3-VL-235B-A22B-Instruct" ], "hardware": { "xeon6": "verified" } }, "recommended_command": { "hardware": "xeon6", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai-cpu:latest-x86_64", "command": "vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \\\n --tensor-parallel-size 1 \\\n --mm-encoder-tp-mode data", "argv": [ "vllm", "serve", "Qwen/Qwen3-VL-30B-A3B-Instruct", "--tensor-parallel-size", "1", "--mm-encoder-tp-mode", "data" ], "docker_command": "docker run --shm-size=16g \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-30B-A3B-Instruct \\\n --tensor-parallel-size 1 \\\n --mm-encoder-tp-mode data", "docker_argv": [ "docker", "run", "--shm-size", "16g", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai-cpu:latest-x86_64", "Qwen/Qwen3-VL-30B-A3B-Instruct", "--tensor-parallel-size", "1", "--mm-encoder-tp-mode", "data" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "Intel", "generation": "cpu", "display_name": "Xeon 6", "description": "Intel Xeon 6 CPU NUMA node · x86 CPU backend", "gpu_count": 4, "vram_gb": 1024, "multi_node": false, "restricted": true }, "alternatives": { "single_node_tep": "/Qwen/Qwen3-VL-30B-A3B-Instruct/strategies/single_node_tep.json", "single_node_dep": "/Qwen/Qwen3-VL-30B-A3B-Instruct/strategies/single_node_dep.json", "multi_node_tp": "/Qwen/Qwen3-VL-30B-A3B-Instruct/strategies/multi_node_tp.json", "multi_node_tp_pp": "/Qwen/Qwen3-VL-30B-A3B-Instruct/strategies/multi_node_tp_pp.json", "multi_node_dep": "/Qwen/Qwen3-VL-30B-A3B-Instruct/strategies/multi_node_dep.json", "multi_node_tep": "/Qwen/Qwen3-VL-30B-A3B-Instruct/strategies/multi_node_tep.json" }, "by_hardware": { "xeon6": "/Qwen/Qwen3-VL-30B-A3B-Instruct/hw/xeon6.json" } }, "model": { "model_id": "Qwen/Qwen3-VL-30B-A3B-Instruct", "min_vllm_version": "0.11.0", "architecture": "moe", "parameter_count": "30B", "active_parameters": "3B", "context_length": 262144, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" }, "extras": [ { "note": "Recommended for offline multimodal inference (image/video pre-processing helpers)", "command": "uv pip install qwen-vl-utils==0.0.14", "optional": true } ] } }, "features": { "text_only": { "description": "Skip loading the vision encoder for text-only workloads — frees memory for KV cache.", "args": [ "--language-model-only" ] }, "encoder_parallel": { "description": "Run the vision encoder in data-parallel mode.", "args": [ "--mm-encoder-tp-mode", "data" ] } }, "opt_in_features": [ "text_only" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 72, "description": "Full precision BF16", "supported_hardware": [ "xeon6" ] } }, "kv_offload_support": { "offloading_cpu": "verified", "offloading_fs": "verified" }, "hardware_overrides": {}, "guide": "## Overview\n\nQwen3-VL-30B-A3B-Instruct is a multimodal Qwen3-VL MoE model. Intel Xeon 6\nCPU serving was validated with BF16.\n\n## Prerequisites\n\n- Hardware: Intel Xeon 6 CPUs\n- vLLM >= 0.11.0\n\n### pip (Intel Xeon 6 CPUs)\n\nFor Intel and AMD x86 CPUs, follow the\n[CPU pre-built wheels](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#pre-built-wheels)\ninstallation instructions.\n\n### Docker (Intel Xeon 6 CPUs)\n\n```bash\ndocker pull vllm/vllm-openai-cpu:latest-x86_64\n```\n\n## Intel Xeon 6\n\n```bash\nvllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \\\n --tensor-parallel-size 1 \\\n --mm-encoder-tp-mode data\n```\n\nDocker (the image entrypoint is `vllm serve`):\n\n```bash\ndocker run \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-30B-A3B-Instruct \\\n --tensor-parallel-size 1 \\\n --mm-encoder-tp-mode data\n```\n\nThe example uses TP=1. Adjust TP/DP based on system topology; NUMA binding is\nintentionally not hard-coded in the recipe.\n\n## Configuration Notes\n\n- TP/DP values and topology-specific CPU binding remain deployment choices.\n\n## Runtime and Platform Tuning\n\nThe following settings are intentionally not prescribed as portable CPU model\ndefaults because they depend on the workload, hardware, or runtime environment:\n\n- `--max-num-batched-tokens`: scheduler/throughput tuning.\n- `--max-num-seqs`: concurrency and scheduler-capacity tuning.\n- `--gpu-memory-utilization`: platform memory-budget tuning.\n- `--no-enable-prefix-caching`: workload/benchmark cache-policy tuning.\n- `VLLM_ENGINE_ITERATION_TIMEOUT_S`: operational runtime timeout.\n\nThese settings may still appear in validated hardware-specific overrides. Tune\nthem at deployment time based on platform resources, workload shape, and\nlatency/throughput goals.\n\n## References\n\n- [Model card](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct)\n- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/Qwen/Qwen3-VL-30B-A3B-Instruct.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/Qwen/Qwen3-VL-30B-A3B-Instruct.yaml", "overlays": [] } }