{
  "hf_id": "Qwen/Qwen-Image-2.1",
  "meta": {
    "title": "Qwen-Image-2.1",
    "slug": "qwen-image-2-1",
    "provider": "Qwen",
    "description": "Unified text-to-image and image-conditioned generation. A 7.1B single-stream DiT with block-causal attention and an exact cross-step prefix KV cache, paired with a Qwen3-VL-8B text encoder and a 16x RGBA autoencoder.",
    "date_added": "2026-09-19",
    "date_updated": "2026-09-19",
    "tasks": [
      "omni"
    ],
    "performance_headline": "One 7.1B DiT serves both text-to-image and editing; the prefix KV cache turns the text and reference-image portion of the sequence into a one-time cost instead of a per-step one",
    "related_recipes": [
      "Qwen/Qwen-Image"
    ],
    "hardware": {
      "gb300": "verified"
    },
    "default_hardware": "gb300"
  },
  "model": {
    "model_id": "Qwen/Qwen-Image-2.1",
    "min_vllm_version": "0.29.0",
    "docker_image": "vllm/vllm-omni:qwen-image21",
    "architecture": "dense",
    "parameter_count": "16B",
    "active_parameters": "16B",
    "context_length": 0,
    "base_args": [],
    "base_env": {},
    "install": {
      "docker": {
        "command": "docker pull vllm/vllm-omni:qwen-image21",
        "note": "vllm-project/vllm-omni#7759 is still under review, so support ships only in the image below. Use Docker until it merges."
      },
      "extras": [
        {
          "note": "vLLM-Omni must be built from PR #7759. Qwen-Image-2.1 is not in a tagged release and not on main either, so a plain clone cannot serve it.",
          "command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && git fetch origin pull/7759/head:qwen-image-2.1 && git checkout qwen-image-2.1 && uv pip install -e ."
        }
      ]
    }
  },
  "omni": {
    "tasks": [
      {
        "id": "t2i",
        "description": "Text to image. 1024x1024 in ~4.5 s on one GB300 at 40 steps.",
        "curl": "curl -X POST http://localhost:8000/v1/images/generations \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"Qwen/Qwen-Image-2.1\",\n    \"prompt\": \"A ceramic teapot on a wooden table, soft window light\",\n    \"size\": \"1024x1024\",\n    \"num_inference_steps\": 40,\n    \"true_cfg_scale\": 1.0,\n    \"seed\": 42\n  }'\n",
        "benchmark": "# No image-generation benchmark exists in vllm bench serve.\n# Time a single request instead:\ntime curl -s -o /dev/null -X POST http://localhost:8000/v1/images/generations \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"Qwen/Qwen-Image-2.1\",\"prompt\":\"a teapot\",\"size\":\"1024x1024\",\"num_inference_steps\":40,\"true_cfg_scale\":1.0,\"seed\":42}'\n"
      },
      {
        "id": "i2i",
        "description": "Image-conditioned generation. Up to 4 reference images per request; a fifth is rejected.",
        "curl": "curl -X POST http://localhost:8000/v1/images/edits \\\n  -F image=@plate.png \\\n  -F 'prompt=Write the words \"FRESH BASIL\" on this plate in dark green lettering' \\\n  -F model=Qwen/Qwen-Image-2.1 \\\n  -F size=1024x1024 \\\n  -F num_inference_steps=40 \\\n  -F true_cfg_scale=1.0 \\\n  -F seed=42\n# The picture comes back base64-encoded at .data[0].b64_json\n#\n# There is a second way in, if you would rather send JSON: the chat\n# endpoint takes image_url content parts and returns the picture at\n# .choices[0].message.content[0].image_url.url\n"
      }
    ]
  },
  "features": {},
  "opt_in_features": [],
  "variants": {
    "default": {
      "label": "BF16",
      "precision": "bf16",
      "vram_minimum_gb": 40,
      "description": "Checkpoint precision. 33.1 GB of weights. Measured 34.0 GB peak on one GB300 at 1024x1024 with --vae-use-tiling (40 steps, CFG off); the contributors report ~40 GB on GB200. Sized by the repo formula (ceil(33.1 GB x 1.2) = 40), which the 34.0 GB measurement sits under."
    },
    "fp8": {
      "precision": "fp8",
      "vram_minimum_gb": 40,
      "description": "Online FP8 scoped to the DiT, holding the composition-sensitive `img_mlp` layers in BF16 (64 of 160 block linears quantized). 32 GB peak against 34 GB in BF16, and no faster. Quantizing the text encoder instead saves more — about 28 GB — and is one more key in the same JSON.",
      "extra_args": [
        "--diffusion-quantization-config",
        "{\"transformer\":{\"method\":\"fp8\",\"ignored_layers\":[\"img_mlp\"]}}"
      ]
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\nvLLM-Omni accelerates Qwen-Image 2.1 through cross-step prefix KV cache reuse and dedicated CUDA Graphs, reducing redundant computation and kernel launch overhead. Request-level and step-level continuous batching improve GPU utilization and throughput, with phase-aware prefill and decode scheduling. It also supports tensor and Ulysses sequence parallelism, distributed VAE decoding with adaptive OOM recovery, FP8 weights and prefix KV storage, and CPU offloading for varying memory budgets.\n\nOne pipeline class, `QwenImage21Pipeline`, serves both text-to-image and\nimage-conditioned generation (editing). Prompt and reference images are encoded\ntogether by a Qwen3-VL vision-language model, so an edit request takes the same\nserving path as plain text-to-image, and there is no separate edit model to load.\n\n| Component | Class | On disk |\n|---|---|---|\n| Text encoder | `Qwen3VLForConditionalGeneration` | bf16, 17.5 GB |\n| DiT | `QwenImage21Transformer2DModel` | bf16, 14.2 GB |\n| Autoencoder | `AutoencoderKLQwenImage21` | fp32, 1.4 GB |\n\nThe autoencoder compresses 16x into a 64-channel latent and carries 4 channels in\nand out, so RGBA transparency survives the round trip.\n\n## Sampling defaults\n\nSample at **40 steps with classifier-free guidance off** — `num_inference_steps=40`\nand `true_cfg_scale=1.0` in the reference implementation's pipeline signature\n([diffusers#14804](https://github.com/huggingface/diffusers/pull/14804)).\n\n```bash\n--num-inference-steps 40 --cfg-scale 1.0\n```\n\n`--cfg-scale` above 1 engages only with a negative prompt; otherwise it is ignored\nwith a warning. When it engages, the DiT runs twice per step, roughly doubling\nlatency.\n\n## Prerequisites\n\nSupport is not in a tagged release and, as of this writing, not yet merged\n(vllm-project/vllm-omni#7759).\n\nThe pipeline itself is known-good: brought up from PR #7759 on one NVIDIA\nGB300, it generated a 1024x1024 PNG in **4.5 s** over 40 steps at **34.0 GB**\npeak.\n\n```bash\ngit clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni\ngit fetch origin pull/7759/head:qwen-image-2.1 && git checkout qwen-image-2.1\nuv venv --python 3.12 --seed && source .venv/bin/activate\nuv pip install vllm==0.29.0 --torch-backend=auto && uv pip install -e .\n```\n\n## Offline inference\n\n```bash\n# text-to-image\npython examples/offline_inference/text_to_image/text_to_image.py \\\n  --model Qwen/Qwen-Image-2.1 \\\n  --prompt \"A ceramic teapot on a wooden table\" \\\n  --output t2i.png --num-inference-steps 40 --cfg-scale 1.0\n\n# image-conditioned\npython examples/offline_inference/image_to_image/image_edit.py \\\n  --model Qwen/Qwen-Image-2.1 --color-format RGBA --seed 42 \\\n  --image input1.png input2.png \\\n  --prompt \"Combine these images into a single scene\" \\\n  --output edit.png --num-inference-steps 40 --cfg-scale 1.0\n```\n\nUp to 4 reference images per request; a fifth is rejected with a 400. Each is resized\nindependently to ~1024x1024 of area, so they may differ in aspect ratio, and each\nis tagged `<image1>`, `<image2>` in the order given, so an ordinal reference in the\nprompt binds a phrase to a photo.\n\n`--color-format RGBA` preserves transparency (the shared example defaults to RGB).\nWidth and height are floored to a multiple of 32, never rejected. When omitted on an edit request\nthe output size derives from the last reference image's aspect ratio at ~1024x1024.\n\n## Online serving\n\n```bash\nvllm serve Qwen/Qwen-Image-2.1 --omni --port 8091\n```\n\nAdd `--step-execution --max-num-seqs 8` to batch requests at the same KV-cache\nphase. Admission is all-or-nothing: once a wave is past its first denoising step, a\nnew arrival waits for it to finish, even if `max-num-seqs` has room.\n\nSend `num_inference_steps` on every request; the server's own default is 50, not\nthe 40 this checkpoint wants. `true_cfg_scale` defaults to 4.0 but only applies\nalongside a `negative_prompt` — send one without the other and you get guidance\nat 4.0 and roughly double the compute.\n\n### Text to image\n\nSend JSON to `/v1/images/generations`. The picture comes back base64-encoded at\n`.data[0].b64_json`.\n\n```bash\ncurl -X POST http://localhost:8091/v1/images/generations \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"Qwen/Qwen-Image-2.1\",\n       \"prompt\": \"A ceramic teapot on a wooden table\",\n       \"size\": \"1024x1024\",\n       \"num_inference_steps\": 40,\n       \"true_cfg_scale\": 1.0,\n       \"seed\": 42}'\n```\n\n### Image editing\n\nSend a **form**, not JSON, to `/v1/images/edits`. The picture is a file upload\nand everything else is a form field. A JSON body to this endpoint resets the\nconnection. Repeat `-F image=@...` for more pictures, up to four.\n\n```bash\ncurl -X POST http://localhost:8091/v1/images/edits \\\n  -F image=@plate.png \\\n  -F 'prompt=Write the words \"FRESH BASIL\" on this plate in dark green lettering' \\\n  -F model=Qwen/Qwen-Image-2.1 \\\n  -F size=1024x1024 \\\n  -F num_inference_steps=40 \\\n  -F true_cfg_scale=1.0 \\\n  -F seed=42\n```\n\nThe reply has the same shape as text to image: the picture is at\n`.data[0].b64_json`.\n\nIf you would rather send JSON, the chat endpoint also accepts pictures. It puts\nthe result somewhere different — `.choices[0].message.content[0].image_url.url`:\n\n```bash\ncurl -X POST http://localhost:8091/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"Qwen/Qwen-Image-2.1\",\n       \"messages\": [{\"role\": \"user\", \"content\": [\n         {\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,<BASE64>\"}},\n         {\"type\": \"text\", \"text\": \"Write the words \\\"FRESH BASIL\\\" on this plate\"}\n       ]}],\n       \"modalities\": [\"image\"],\n       \"extra_body\": {\"size\": \"1024x1024\", \"num_inference_steps\": 40,\n                      \"true_cfg_scale\": 1.0, \"seed\": 42}}'\n```\n\n## Prefix KV cache\n\nOn automatically; rebuilt per generation, so nothing leaks between requests. The\nprompt and any reference images are cached after the first denoising step and\nreused for the rest, which is exact rather than approximate.\n\nIt pays off in proportion to how much prefix there is: repeating a prompt costs\nabout a third of the first submission, four reference images leave only about a\nfifth of the sequence to recompute, and plain short-prompt text-to-image has\nalmost nothing to cache. Changing the tail of a prompt loses the saving.\n\nFor long-prompt or multi-image workloads it can be stored in FP8 via the stage\n`extras` key `prefix_kv_cache_dtype` — on the command line through\n`--stage-overrides '{\"0\":{\"extras\":{\"prefix_kv_cache_dtype\":\"fp8_v\"}}}'`, or as a\nstage field in a deploy YAML. `\"fp8_v\"` quantizes V only, `\"fp8\"` quantizes K too — the contributors\nmeasured ~41 dB and ~35 dB PSNR against BF16. A quantized cache returns less of\nthe speed-up than one in the native dtype, and is not CUDA-graph capturable, so\nthose requests fall back to eager decode.\n\n## CUDA graphs\n\nFixed-shape decode steps are captured automatically (`enforce_eager=False` by\ndefault); prefill stays eager. Each image layout gets its own entry and prefix K/V\nis copied into static buffers before replay, which costs memory. TP, SP/ring, HSDP,\noffload and cache hooks, compiled blocks, quantized KV caches, dynamic LoRA and\npadded text masks each force eager decode. Disable with `--enforce-eager`; the AR\nengine's `compilation_config.cudagraph_mode` does not reach this path.\n\nExpect the first request after startup to cost about 1.3x a later one.\n`--enforce-eager` does not remove that cost, so it is not graph capture and\ndisabling graphs will not flatten startup.\n\n## Quantization\n\nOnline FP8, per component:\n\n```python\nfrom vllm_omni import Omni\nomni = Omni(model=\"Qwen/Qwen-Image-2.1\", quantization_config={\n    \"transformer\": {\"method\": \"fp8\", \"ignored_layers\": [\"img_mlp\"]},\n    \"text_encoder\": {\"method\": \"fp8\"},\n})\n```\n\nOnly the DiT's block-internal linears are eligible; boundary projections stay\nBF16. Measured by the vLLM-Omni contributors on GB200 at 1024x1024, 50 steps:\n\n| Config | Quantized | Avg PSNR vs BF16 | Peak memory |\n|---|---|---|---|\n| BF16 | 0 / 160 | — | 40.0 GB |\n| FP8, all layers | 160 / 160 | 26.1 dB | 33.4 GB |\n| FP8, `ignored_layers=[\"img_mlp\"]` | 64 / 160 | 29.7 dB | 38.3 GB |\n\nMeasured here on one GB300 at 1024x1024, 40 steps, same prompt and seed in each\ncase:\n\n| Config | Peak memory | Time per image |\n|---|---|---|\n| BF16 | 34.0 GB | 3282-4492 ms |\n| DiT FP8, `img_mlp` kept BF16 | 32.0-32.7 GB | 3356-4706 ms |\n| Text-encoder FP8 | **27.5-28.1 GB** | 3427-4770 ms |\n\nThe text encoder is the bigger saving by a wide margin: about 6 GB against about\n2 GB. Neither runs faster.\n\nKeeping `img_mlp` in BF16 protects composition but gives back most of the saving.\nThe larger win is the text encoder: in a separate GB200 run, peak memory went ~41.0\nGiB in BF16 to ~34.4 GiB with text-encoder FP8, and ~28.1 GiB with the DiT too. Its\nvision tower and unused `lm_head` are excluded by construction, but reference-image\ntokens still cross the quantized language-model linears, so text-encoder FP8 does\nreach the edit path. On Blackwell FP8 buys memory, not speed.\n\n## Limitations\n\n- Cache backends (`cache_dit`, `tea_cache`) unsupported — the model's own prefix\n  cache conflicts with step-skipping hooks.\n- Sequence parallelism is Ulysses only — no ring attention, no pipeline parallelism.\n- True CFG only; there is no `guidance_scale`.\n- `/v1/images/edits` takes **multipart/form-data**, not JSON. Send the picture\n  as a file upload and the settings as form fields. Posting a JSON body to it\n  resets the connection, which is easy to mistake for a broken endpoint.\n- Passing a picture as a `data:` URL in the `url` form field is limited to\n  **1024 KB per part**. A 512x512 PNG from this model is about 1 MB, so it\n  usually fails with `Part exceeded maximum size of 1024KB`. File uploads have\n  no such limit. Base64 inside a form field only suits small images.\n- `/v1/chat/completions` with `image_url` content parts works too, and returns\n  the picture at `.choices[0].message.content[0].image_url.url`.\n- NVIDIA GPUs only.\n- VAE tiling (`--vae-use-tiling`) decodes in 512px tiles at 384px stride, halving\n  on CUDA OOM down to 128px; tile sizes are not exposed as engine arguments.\n\n## References\n\n- [In-repo recipe](https://github.com/vllm-project/vllm-omni/blob/qwen-image-2.1/recipes/Qwen/Qwen-Image-2.1.md)\n- [Feature support matrix](https://github.com/vllm-project/vllm-omni/blob/qwen-image-2.1/docs/user_guide/diffusion_features.md)\n",
  "source": {
    "source": "vllm-omni",
    "repo": "vllm-project/vllm-omni",
    "ref": "main",
    "sha": "cda4a1ef83b295ba332146a7709e6277725edc9f",
    "path": "recipes/models/Qwen/Qwen-Image-2.1.yaml",
    "edit_url": "https://github.com/vllm-project/vllm-omni/edit/main/recipes/models/Qwen/Qwen-Image-2.1.yaml",
    "overlays": []
  }
}