# Qwen/Qwen3-1.7B — vLLM serving recipe

> Qwen3 1.7B dense model with hybrid thinking/non-thinking modes, validated for BF16 serving on Intel Xeon 6.

Configuration: hardware `xeon6` (4× Xeon 6) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/Qwen/Qwen3-1.7B
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/Qwen/Qwen3-1.7B.json?hardware=xeon6

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `xeon6`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.8.5.

Docker image: `vllm/vllm-openai-cpu:latest-x86_64`

## Serve

```bash
vllm serve Qwen/Qwen3-1.7B \
  --tensor-parallel-size 1 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3
```

Docker equivalent:

```bash
docker run --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-1.7B \
  --tensor-parallel-size 1 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Qwen3-1.7B is a compact dense Qwen3 model with hybrid thinking and non-thinking
modes. This recipe includes an Intel Xeon 6 CPU configuration validated with
BF16 serving.

### Prerequisites

- Hardware: 1x Xeon 6 CPUs
- vLLM >= 0.8.5

#### pip (Intel Xeon 6 CPUs)

For Intel and AMD x86 CPUs, follow the
[CPU pre-built wheels](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#pre-built-wheels)
installation instructions.

#### Docker (Intel Xeon 6 CPUs)

```bash
docker pull vllm/vllm-openai-cpu:latest-x86_64
```

### Intel Xeon 6

Choose the tensor parallel size based on the system topology and deployment
requirements. Add `--tensor-parallel-size <N>` when needed; the recipe does
not prescribe a Xeon 6 TP value or hard-code NUMA node IDs.

```bash
vllm serve Qwen/Qwen3-1.7B \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.8
```

Docker:

```bash
docker run \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-1.7B \
  --tensor-parallel-size 1 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3
```

### Client Usage

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
response = client.chat.completions.create(
    model="Qwen/Qwen3-1.7B",
    messages=[{"role": "user", "content": "Explain tensor parallelism briefly."}],
)
print(response.choices[0].message.content)
```

### References

- [Model card](https://huggingface.co/Qwen/Qwen3-1.7B)
- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)