{
  "hf_id": "Qwen/Qwen3-14B",
  "meta": {
    "title": "Qwen3-14B",
    "slug": "qwen3-14b",
    "provider": "Qwen",
    "description": "Qwen3 14B dense model with hybrid thinking/non-thinking modes, validated for Intel Xeon 6 CPU serving.",
    "date_added": "2026-08-17",
    "date_updated": "2026-09-12",
    "tasks": [
      "text"
    ],
    "performance_headline": "Validated on Intel Xeon 6 CPU with BF16 serving",
    "related_recipes": [
      "Qwen/Qwen3-4B",
      "Qwen/Qwen3-32B"
    ],
    "hardware": {
      "xeon6": "verified"
    }
  },
  "recommended_command": {
    "hardware": "xeon6",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai-cpu:latest-x86_64",
    "command": "vllm serve Qwen/Qwen3-14B \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser hermes",
    "argv": [
      "vllm",
      "serve",
      "Qwen/Qwen3-14B",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "hermes"
    ],
    "docker_command": "docker run --shm-size=16g \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-14B \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser hermes",
    "docker_argv": [
      "docker",
      "run",
      "--shm-size",
      "16g",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai-cpu:latest-x86_64",
      "Qwen/Qwen3-14B",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "hermes"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "Intel",
      "generation": "cpu",
      "display_name": "Xeon 6",
      "description": "Intel Xeon 6 CPU NUMA node · x86 CPU backend",
      "gpu_count": 4,
      "vram_gb": 1024,
      "multi_node": false,
      "restricted": true
    },
    "alternatives": {
      "multi_node_tp": "/Qwen/Qwen3-14B/strategies/multi_node_tp.json"
    },
    "by_hardware": {
      "xeon6": "/Qwen/Qwen3-14B/hw/xeon6.json"
    }
  },
  "model": {
    "model_id": "Qwen/Qwen3-14B",
    "min_vllm_version": "0.8.5",
    "architecture": "dense",
    "parameter_count": "14B",
    "active_parameters": "14B",
    "context_length": 40960,
    "base_args": [],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "Enable automatic tool choice with the Hermes parser",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "hermes"
      ]
    },
    "reasoning": {
      "description": "Enable Qwen3 thinking mode with the Qwen3 reasoning parser",
      "args": [
        "--reasoning-parser",
        "qwen3"
      ]
    },
    "long_context": {
      "description": "YaRN RoPE scaling to the model card's validated 131,072-token context. Static YaRN holds the scaling factor constant regardless of input length, so leave this off unless you actually serve long prompts — it can cost quality below 32K.",
      "args": [
        "--hf-overrides",
        "{\"max_position_embeddings\":131072,\"rope_parameters\":{\"rope_type\":\"yarn\",\"factor\":4.0,\"original_max_position_embeddings\":32768}}",
        "--max-model-len",
        "131072"
      ]
    }
  },
  "opt_in_features": [
    "reasoning",
    "long_context"
  ],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 34,
      "description": "Full precision BF16",
      "supported_hardware": [
        "xeon6"
      ]
    },
    "fp8": {
      "model_id": "Qwen/Qwen3-14B-FP8",
      "precision": "fp8",
      "vram_minimum_gb": 17,
      "description": "Qwen official FP8 checkpoint",
      "json": "/Qwen/Qwen3-14B-FP8.json"
    },
    "awq": {
      "model_id": "Qwen/Qwen3-14B-AWQ",
      "precision": "int4",
      "vram_minimum_gb": 10,
      "description": "Qwen official AWQ 4-bit checkpoint",
      "supported_hardware": [
        "xeon6"
      ],
      "extra_args": [
        "--quantization",
        "awq"
      ],
      "json": "/Qwen/Qwen3-14B-AWQ.json"
    }
  },
  "kv_offload_support": {
    "offloading_cpu": "verified",
    "offloading_fs": "verified"
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\nQwen3-14B is a Qwen3 dense model with hybrid thinking and non-thinking modes.\nIntel Xeon 6 CPU serving was validated with BF16.\n\n## Prerequisites\n\n- Hardware: Intel Xeon 6 CPUs\n- vLLM >= 0.8.5\n\n### pip (Intel Xeon 6 CPUs)\n\nFollow the [CPU pre-built wheels](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#pre-built-wheels)\ninstallation instructions.\n\n### Docker (Intel Xeon 6 CPUs)\n\n```bash\ndocker pull vllm/vllm-openai-cpu:latest-x86_64\n```\n\n## Intel Xeon 6\n\nChoose tensor parallelism based on the system topology; the recipe does not\nprescribe a fixed TP/DP layout or hard-code NUMA node IDs.\n\n```bash\nvllm serve Qwen/Qwen3-14B\n```\n\nDocker:\n\n```bash\ndocker run \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-14B \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser hermes\n```\n\n## Configuration Notes\n\n- TP/DP values and explicit CPU bindings are intentionally not prescribed.\n- The `qwen3` reasoning parser is available in vLLM 0.9.0 and later.\n\n## Runtime and Platform Tuning\n\nThe following settings are intentionally not prescribed as portable CPU model\ndefaults because they depend on the workload, hardware, or runtime environment:\n\n- `--max-num-batched-tokens`: scheduler/throughput tuning.\n- `--max-num-seqs`: concurrency and scheduler-capacity tuning.\n- `--gpu-memory-utilization`: platform memory-budget tuning.\n- `--no-enable-prefix-caching`: workload/benchmark cache-policy tuning.\n- `VLLM_ENGINE_ITERATION_TIMEOUT_S`: operational runtime timeout.\n\nThese settings may still appear in validated hardware-specific overrides. Tune\nthem at deployment time based on platform resources, workload shape, and\nlatency/throughput goals.\n\n## References\n\n- [Model card](https://huggingface.co/Qwen/Qwen3-14B)\n- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "389ffebe555705c85337025d5df108d0384d365e",
    "path": "recipes/models/Qwen/Qwen3-14B.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/Qwen/Qwen3-14B.yaml",
    "overlays": []
  }
}