# stabilityai/stable-diffusion-3.5-medium — vLLM serving recipe

> Stability AI's Stable Diffusion 3.5 text-to-image family (medium 2.5B, large 8.1B, large-turbo) via vLLM-Omni with Cache-DiT acceleration

Configuration: hardware `h200` (8× H200) · variant `default` (BF16) · task `t2i`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/stabilityai/stable-diffusion-3.5-medium
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/stabilityai/stable-diffusion-3.5-medium.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `large`, `large_turbo`
- `task`: `t2i`

## Install

Requires vLLM >= 0.12.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Extra dependencies:

- Pin vllm==0.12.0 for Stable Diffusion 3.5

```bash
uv pip install vllm==0.12.0
```

- vllm-omni provides the image generation backend

```bash
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
vllm serve stabilityai/stable-diffusion-3.5-medium \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Stable Diffusion 3.5 text-to-image generation models, served via **vLLM-Omni** with
optional **Cache-DiT** acceleration.

Supported variants:
- `stabilityai/stable-diffusion-3.5-medium` — 2.5B params
- `stabilityai/stable-diffusion-3.5-large` — 8.1B params
- `stabilityai/stable-diffusion-3.5-large-turbo` — 8.1B params (timestep-distilled for few-step inference)

### Prerequisites

- vLLM-Omni on top of vLLM 0.12.0
- `diffusers` library

### Installation

```bash
uv venv
source .venv/bin/activate
uv pip install vllm==0.12.0
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

#### AMD ROCm (MI300X, MI325X, MI355X)

> **Note:** The vLLM wheel for ROCm requires Python 3.12, ROCm 7.2.1, and glibc >= 2.35.
> If your environment does not meet these requirements, please use the Docker-based setup.

```bash
uv venv --python 3.12
source .venv/bin/activate
VLLM_OMNI_TARGET_DEVICE=rocm uv pip install git+https://github.com/vllm-project/vllm-omni.git vllm==0.20.0+rocm721 --extra-index-url https://wheels.vllm.ai/rocm/0.20.0/rocm721
```

### Python Usage

```python
from vllm_omni.entrypoints.omni import Omni

omni = Omni(model="stabilityai/stable-diffusion-3.5-medium")

images = omni.generate(
    prompt="a cat wearing sunglasses, cyberpunk style",
    negative_prompt="blurry, low quality",
    height=1024, width=1024,
    num_inference_steps=28,
    guidance_scale=7.5,
    num_outputs_per_prompt=2,
)
```

### CLI Usage

```bash
python examples/offline_inference/text_to_image/text_to_image.py \
  --model stabilityai/stable-diffusion-3.5-medium \
  --prompt "a cat wearing sunglasses, cyberpunk style" \
  --negative-prompt "blurry, low quality" \
  --height 1024 --width 1024 \
  --num-inference-steps 28 \
  --guidance-scale 7.5
```

### Cache-DiT Acceleration

Enable caching for significant speed-ups:

```python
omni = Omni(
    model="stabilityai/stable-diffusion-3.5-medium",
    cache_backend="cache_dit",
    cache_config={
        "Fn_compute_blocks": 8,
        "Bn_compute_blocks": 0,
        "max_warmup_steps": 4,
        "residual_diff_threshold": 0.12,
    },
)
```

### Key Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `height` | 1024 | Image height (multiples of 16) |
| `width` | 1024 | Image width (multiples of 16) |
| `num_inference_steps` | 28 | Denoising steps |
| `guidance_scale` | 1.0 | Classifier-free guidance scale |

### References

- [SD3.5 Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium)
- [SD3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large)
- [SD3.5 Large Turbo](https://huggingface.co/stabilityai/stable-diffusion-3.5-large-turbo)
- [Cache-DiT Acceleration](https://github.com/vipshop/cache-dit)