# MiniMaxAI/MiniMax-H3 — vLLM serving recipe

> Open-weight general-purpose multimodal generation model — jointly generates 24 FPS video with native stereo audio from text, image, video, and audio references, served via vLLM-Omni

Configuration: hardware `b300` (8× B300) · variant `default` (BF16) · task `t2va`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/MiniMaxAI/MiniMax-H3
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/MiniMaxAI/MiniMax-H3.json?hardware=b300

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `dgx_spark_gb10`, `rtx_pro_6000_2x`, `rtx_pro_5000_4x`, `rtx_4090_2x`, `rtx_5090_2x`, `mi300x`, `mi325x`, `mi350x`, `mi355x`, `ascend_950pr`
- `variant`: `default`
- `task`: `t2va`, `fl2va_first`, `fl2va_first_last`, `ref_image`, `ref_image_audio`, `ref_video_mixed`

## Install

Requires vLLM >= 0.26.0.

H3 also needs current vLLM-Omni main for the modular FL2VA/Ref2VA pipeline — see the extra install block below.

```bash
uv venv
source .venv/bin/activate
uv pip install vllm==0.26.0
```

Extra dependencies:

- Gated checkpoint — request access on the model card, then download it locally. Serve the root directory: both DiTs remain in subdirectories while the modular service shares the encoder and VAEs.

```bash
hf download MiniMaxAI/MiniMax-H3 --local-dir /path/to/MiniMax-H3
```

- pip path only: install current vLLM-Omni main for the modular MiniMax-H3 pipeline.

```bash
git clone --depth 1 https://github.com/vllm-project/vllm-omni.git /path/to/vllm-omni && cd /path/to/vllm-omni && uv pip install -e .
```

- Optional FastH3 adapters: download the Dense / Data-Free baseline and the recommended VSA / Data-Free artifact under the already-mounted H3 root. FastH3 preview v1 serves T2VA only and is fused at startup. (optional)

```bash
hf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
  dense-datafree/adapter_model.safetensors \
  vsa-datafree/adapter_model.safetensors \
  --local-dir /path/to/MiniMax-H3/FastH3-LoRA
```

- Optional LightX2V Turbo few-step LoRA: download one Diffusers-layout artifact under the already-mounted H3 root. The filename carries the sampling contract, so do not rename it; the _comfyui_ exports are not supported. (optional)

```bash
hf download lightx2v/Minimax-h3-Turbo \
  minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors \
  --local-dir /path/to/MiniMax-H3/Turbo-LoRA
```

- Optional CUDA-only FastH3 VSA kernel for the pip path. --no-deps preserves the PyTorch/CUDA stack managed by vLLM. The tested B300/SM103 route uses the Triton fallback with FASTVIDEO_VSA_SM100A=0. (optional)

```bash
uv pip install --no-deps fastvideo-kernel==0.3.5
```

- Optional pip extra for explicitly comparing FA4 on datacenter Blackwell. The generated B300/GB200 route uses the TRTLLM_ATTN default; RTX PRO 5000 and the two-GPU RTX routes use CUDNN_ATTN. (optional)

```bash
cd /path/to/vllm-omni && uv pip install -e '.[fa4]'
```

- Docker path only: clone current vLLM-Omni main to the path mounted by the generated command so it overrides the older package in the image.

```bash
git clone --depth 1 https://github.com/vllm-project/vllm-omni.git /path/to/vllm-omni
```

- pip path only: ffmpeg and ffprobe must be on PATH for reference-video preparation and MP4 video+audio muxing. The official Docker images already bundle them.

```bash
sudo apt-get install -y ffmpeg
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800
export FLASHINFER_DISABLE_VERSION_CHECK=1
vllm serve /path/to/MiniMax-H3 \
  --trust-remote-code \
  --host 0.0.0.0 \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --port 8000 \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

MiniMax H3 is an **open-weight, general-purpose multimodal generation model**. Rather than
being confined to one specialized task — generate, edit, or reference — H3 reads a
multimodal context that mixes text, images, video, and audio together, interprets the
creative intent as a whole, and produces coherent audio-visual output end to end.

Architecturally it is a **CFG-distilled joint video/audio diffusion transformer** served
through vLLM-Omni's OpenAI-compatible `/v1/videos` API. Every request returns a single MP4
containing H.264 video and **native stereo audio** — the audio is generated jointly with
the video by the same DiT, not dubbed on afterwards.

Three capability areas the model targets:

- **Commercial-grade generation** — film and entertainment, advertising and branding,
  e-commerce, gaming; dynamic typography, VFX, product showcases, UI/UX motion design.
- **Native multimodal understanding + generation** — interprets characters, motion, sound,
  emotion, cinematography, and visual style across mixed references, then combines them.
- **Precise multimodal editing and control** — iterative refinement of characters, objects,
  scenes, sound, and rhythm with strong instruction following.

The checkpoint contains two DiT backbones that share the encoder and VAEs:

| DiT | Tasks | Conditioning |
|---|---|---|
| `FL2VA` | `t2va`, `fl2va` | text only, or text + first/last frame |
| `Ref2VA` | `ref2va` | text + omni references (images, video clips, audio clips) |

Since vLLM-Omni [#5720](https://github.com/vllm-project/vllm-omni/pull/5720), serving
`/path/to/MiniMax-H3` initializes both DiTs by default, while the Qwen3-VL encoder and
video/audio VAEs stay shared. One server can therefore handle all three task types.
Add `--task-type fl2va` or `--task-type ref2va` only when the hardware profile needs to
load one DiT to save GPU or host memory.

Components (BF16): two 52-block joint video/audio DiTs (66.3 GB each), a shared
Qwen3-VL layer-50 text/vision encoder (51.5 GB), a video VAE (~10 GB), and an audio
VAE (~0.6 GB).

### Model specifications

| | |
|---|---|
| Output duration | 4–15 seconds |
| Frame rate | 24 FPS (fixed) |
| Audio | Native stereo, on every generation |
| Resolution — 1440p (2K) | Short edge 1440 px for 16:9 … 9:16; wider formats ≈3.7 MP (e.g. 2976×1248 at 21:9) |
| Resolution — 768p | Short edge 768 px for 16:9 … 9:16; wider formats ≈1 MP (e.g. 1536×672 at 21:9). Upscalable to 1440p. *(mode coming soon)* |
| Aspect ratio | First/Last Frame follows the uploaded image. Text-to-Video and Omni Reference select from 21:9, 16:9, 4:3, 1:1, 3:4, 9:16 — Omni Reference also offers Auto |
| Prompt length | Up to 7,000 characters |

#### Input modes and limits

**First/Last Frame** — 0, 1, or 2 images. Edge lengths in `[256, 5760]`, aspect ratio 5:2
to 2:5. With no image, H3 falls back to Text-to-Video.

**Omni Reference** — up to 9 images; up to 3 video clips (2–15 s each, 15 s total); up to
3 audio clips (2–15 s each, 15 s total). Edge lengths in `[256, 5760]`, video aspect ratio
5:2 to 2:5. **Audio must be paired with at least one image or video** — it cannot be the
sole reference. Mixed inputs cap at 12 files total; with none, H3 falls back to
Text-to-Video.

| | Formats | Per-file size |
|---|---|---|
| Video | H.264/AVC, H.265/HEVC (embedded audio AAC or MP3) | 50 MB |
| Image | JPG, JPEG, PNG, WEBP, HEIC, HEIF | 30 MB |
| Audio | WAV, MP3 | 15 MB |

Limits are per file, not per combined upload. Keep the whole request body under 64 MB —
prefer URL-based media over inlined uploads for anything large.

### Prerequisites

H3 can be served directly from the gated HF ID or from a local root directory. The local root contains both DiT
subdirectories plus the shared encoder and VAEs. Request access on the gated model card,
then download the complete repository:

```bash
hf download MiniMaxAI/MiniMax-H3 --local-dir /path/to/MiniMax-H3
```

That yields `/path/to/MiniMax-H3/FL2VA`, `/path/to/MiniMax-H3/Ref2VA`, and the
shared component directories. Serve `/path/to/MiniMax-H3` itself and substitute your own
`--local-dir` throughout the commands below.

`ffmpeg` and `ffprobe` must be on `PATH` — they are used for reference-video preparation
and MP4 muxing.

### Installation

#### Docker (recommended)

The image bundles H3 handlers and media dependencies. The published tag predates the
modular pipeline merged in vLLM-Omni #5720, so clone current `main` before starting:

```bash
docker pull vllm/vllm-omni:minimax-h3
git clone --depth 1 https://github.com/vllm-project/vllm-omni.git /path/to/vllm-omni
```

The command builder's Docker mode binds both the root checkpoint and source checkout,
then prepends the checkout to `PYTHONPATH`:

```bash
-v /path/to/MiniMax-H3:/path/to/MiniMax-H3:ro
-v /path/to/vllm-omni:/path/to/vllm-omni:ro
-e PYTHONPATH=/path/to/vllm-omni
```

#### pip

H3 support ships in vLLM-Omni rather than the `vllm` wheel, so the pip path needs a
current source checkout:

```bash
uv venv
source .venv/bin/activate
uv pip install vllm==0.26.0
git clone --depth 1 https://github.com/vllm-project/vllm-omni.git /path/to/vllm-omni
cd /path/to/vllm-omni
uv pip install -e .
```

The generated datacenter Blackwell route uses the default `TRTLLM_ATTN`; RTX PRO 5000 and the two-GPU RTX profiles use `CUDNN_ATTN`. These routes need no
FlashAttention-4 install. Install the optional extra only when explicitly comparing FA4:
`uv pip install -e '.[fa4]'`.

### Launch — two 24/32 GB GPUs with distributed layerwise offload

vLLM-Omni includes a low-memory TP2 path for H3. Install the current vLLM-Omni
source checkout as shown above.

The two-GPU profile combines TP2 with distributed layerwise offload (DLO). Each rank
keeps its rank-local TP shard in pinned host memory; `--dlo-no-use-allgather` streams
those shards directly instead of reconstructing full DiT blocks on every rank. A
configurable prefix of DiT blocks stays resident for the complete denoise stage, then
releases its HBM before VAE decode.

Select the profile first. RTX 5090 uses 20 resident DiT layers at the released
1344×768 shape; RTX 4090 keeps more HBM headroom with 12 resident layers and a
1024×576 starting shape:

```bash
# 2× RTX 5090 (32 GB)
export DLO_RESIDENT_LAYERS=20 WIDTH=1344 HEIGHT=768

# Or, 2× RTX 4090 (24 GB)
# export DLO_RESIDENT_LAYERS=12 WIDTH=1024 HEIGHT=576
```

Load only the FL2VA DiT on these memory-constrained profiles:

```bash
CUDA_VISIBLE_DEVICES=0,1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --task-type fl2va \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 2 \
  --tensor-parallel-size 2 \
  --usp 1 \
  --ring 1 \
  --text-encoder-tp-size 2 \
  --vae-patch-parallel-size 2 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --enable-distributed-layerwise-offload \
  --dlo-no-use-allgather \
  --dlo-resident-layers "${DLO_RESIDENT_LAYERS}" \
  --enforce-eager \
  --diffusion-attention-backend CUDNN_ATTN
```

The resident count changes placement and transfer frequency only; it does not quantize
or change the BF16/FP32 denoise math. Use a **384 GiB-class host** for the pinned model
shards and offload buffers.

Wait for `Application startup complete`, then verify health and run one full 50-step
T2VA request:

```bash
curl --fail http://127.0.0.1:8000/health

curl --fail-with-body -sS \
  -X POST http://127.0.0.1:8000/v1/videos/sync \
  -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \
  -F "width=${WIDTH}" \
  -F "height=${HEIGHT}" \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":5.0,"audio_flow_shift":3.0}' \
  -o minimax-h3-t2va.mp4

ffprobe -v error -show_entries \
  stream=index,codec_name,width,height,r_frame_rate,sample_rate,channels \
  -of json minimax-h3-t2va.mp4
ffmpeg -v error -i minimax-h3-t2va.mp4 \
  -map 0:v:0 -map 0:a:0 -f null -
```
**Note:** If using 2× RTX 4090, the curl request must include -F 'aspect_ratio=16:9' to succeed.

A successful output has H.264 video at 24 FPS and stereo AAC audio at 32 kHz, and the
final `ffmpeg` command exits with status 0. This capacity profile intentionally loads one
DiT. Restart the same root-model command with `--task-type ref2va` for Ref2VA; remove
`--task-type` only after confirming that the combined modular service fits the target.

The same root path is used for both task families; only `--task-type` changes on these
capacity profiles.

### Launch — four GPUs (validated best practice)

On B300/GB200, the no-offload root service uses Ulysses SP4, tiled VAE patch
parallelism 4, regional `torch.compile`, dense BF16 `TRTLLM_ATTN`, and Ring/TP1. Both DiTs load
once while the encoder and VAEs remain shared.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 \
FLASHINFER_DISABLE_VERSION_CHECK=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling
```

Omitting the attention flag is intentional: current datacenter Blackwell builds default to
dense BF16 `TRTLLM_ATTN`. Install `[fa4]` and add `--diffusion-attention-backend FLASH_ATTN`
only for an explicit FA4 comparison.

#### Optional lossy attention: SAGE + Skip-Softmax

`TRTLLM_ATTN` offers two **lossy** optimizations for the long main DiT attention sequence.
Both work under the pure Ulysses parallelism of the four-GPU profile (`--usp 4 --ring 1`) and
are datacenter-Blackwell only. Append the following to the four-GPU command above to enable
both:

```bash
--diffusion-attention-config '{
  "default": {
    "backend": "TRTLLM_ATTN",
    "quant": {"dtype_qk": "fp8_e4m3", "q_block_size": 1, "k_block_size": 16},
    "skip_softmax": {"threshold": 0.05, "disabled_until_timestep": 0.97}
  },
  "per_role": {
    "minimax_h3.token_refiner": {"backend": "TRTLLM_ATTN"}
  }
}'
```

- **SAGE** quantizes Q/K to `fp8_e4m3` for `QK^T`; P and V are always FP8 in this kernel.
  B200 additionally supports `int8` Q/K, which preserves accuracy better than FP8.
- **Skip-Softmax** skips the Softmax and `PV` work of KV tiles whose scores fall far below
  the running row maximum. `threshold=0.05` with `disabled_until_timestep=0.97` is a
  **conservative** setting: a low threshold skips only clearly negligible tiles, and the
  cutoff keeps the early high-noise steps dense — H3's default video flow shift of 12 keeps
  the normalized timestep above `0.97` for the first 14 of 49 denoiser forwards at 50 steps.
  Raise `threshold` or lower `disabled_until_timestep` for more speedup once quality is
  verified.
- The `per_role` entry keeps the short token-refiner attention dense; a per-role spec does
  not inherit `quant` or `skip_softmax` from `default`.

Both optimizations trade fidelity for speed, and their effects compound. The measured B300
speedup and LPIPS for each mode and for the combination are in the vLLM blog's
[Quantized and Sparse Attention](https://vllm.ai/blog/2026-09-01-minimax-h3-production-serving#quantized-and-sparse-attention-in-trtllm_attn)
section as a reference; tune the settings according to the accuracy and speed required. See
the
[TRTLLM attention guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/attention_backends/trtllm/)
for the full configuration reference.

No restart is required: `extra_params.task` routes T2VA/FL2VA requests to the FL2VA
DiT and Ref2VA requests to the Ref2VA DiT.

#### FastH3 model family

FastH3 is a family of post-trained models based on MiniMax-H3. It generates
video with stereo audio in four or eight steps. Choose a
starting point for your input and hardware:

| Input | Hardware | Model |
|---|---|---|
| Prompt and reference image | RTX 5090, DGX Spark, RTX PRO 6000 Blackwell | [Omni NVFP4 QAD](https://huggingface.co/FastVideo/FastH3-Omni-NVFP4-QAD), single-card guide below |
| Prompt and reference image | RTX 4090 | [Omni BF16](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1) with FP8 enabled, single-card guide below |
| Prompt and reference image | Eight GPUs | Omni BF16, eight-GPU guide below |
| Prompt only | Eight GPUs | [V2](https://huggingface.co/FastVideo/FastVideo-FastH3-8-Step-V2) (experimental) or [four-step adapter](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA) |

The command builder above serves native MiniMax-H3. Use the FastH3 setup
and commands below for these models. Select one model when starting the server.
The single-card profiles are configuration guidance; this documentation change
does not independently qualify them with GPU inference.

##### Install FastH3

Use Linux, Python 3.12, a CUDA 13-compatible NVIDIA driver, and
[uv](https://docs.astral.sh/uv/getting-started/installation/). The eight-step
models currently need the source branch from
[vLLM-Omni #9167](https://github.com/vllm-project/vllm-omni/pull/9167):

```bash
sudo apt-get update
sudo apt-get install -y git ffmpeg

git clone --branch pr/fasth3-omni-upstream https://github.com/lishunyang12/vllm-omni.git vllm-omni-fasth3
cd vllm-omni-fasth3
uv venv --python 3.12
source .venv/bin/activate
uv pip install vllm==0.32.0 --torch-backend=cu130
uv pip install -e .
```

On DGX Spark, run in the native `aarch64` environment. If Hugging Face
requests access, accept the model's access terms and run `hf auth login`.
Run the following server commands from the `vllm-omni-fasth3` directory.

##### FastH3 RefOmni Single Card

**RTX 5090 (32 GB), DGX Spark (128 GB), or RTX PRO 6000 Blackwell (96 GB):**

```bash
CUDA_VISIBLE_DEVICES=0 VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=7200 \
vllm serve FastVideo/FastH3-Omni-NVFP4-QAD --omni \
  --config recipes/FastVideo/fasth3-omni-quantized.yaml
```

This uses the smaller NVFP4 model and fast 8-layer video decoder. Layer
offload reduces GPU memory use.

**RTX 4090 (24 GB):**

```bash
uv pip install -e '.[vsa]'

CUDA_VISIBLE_DEVICES=0 VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=7200 \
vllm serve FastVideo/FastVideo-FastH3-Omni-8-Step-V1 --omni \
  --config recipes/FastVideo/fasth3-omni-rtx4090.yaml
```

The first start downloads the model; the 4090 also converts eligible
weights to FP8 while loading. Later starts reuse the download cache.
Keep the terminal open and wait for `Application startup complete`.

**Generate.** Save your image as `reference.png`. In a second terminal:

```bash
curl --fail-with-body -sS http://localhost:8000/v1/videos/sync \
  -F 'prompt=The person in the reference image smiles and waves at the camera. Birds chirp as the person says, "Welcome to the garden!"' \
  -F 'input_reference=@reference.png' \
  -F 'size=832x480' -F 'seconds=5' -F 'seed=1234' \
  -o output.mp4
```

Open `output.mp4` to play the video with stereo audio. For 768P, change
`size=832x480` to `size=1344x768`. If memory is tight, start at 480P and close
other GPU workloads. For multiple images, video, or audio references, see
the [H3 reference API](https://github.com/lishunyang12/vllm-omni/blob/pr/fasth3-omni-upstream/recipes/MiniMaxAI/MiniMax-H3.md).

**Optional settings:**

- **FP8 model:** if you have access to [FastH3-Omni-FP8](https://huggingface.co/FastVideo/FastH3-Omni-FP8),
  replace the model ID in the Blackwell command with `FastVideo/FastH3-Omni-FP8`.
  A local model directory also works. This release also requires Blackwell
  because its text encoder uses NVFP4.
- **36-layer decoder:** in `recipes/FastVideo/fasth3-omni-quantized.yaml`, change
  `additional-config.video_vae_decoder.subfolder` from `fast-8L` to
  `quality-36L`, then restart. Both decoders come from
  [FastH3-NVFP4-VAE-Decoder](https://huggingface.co/FastVideo/FastH3-NVFP4-VAE-Decoder).
- **DGX Spark:** remove `diffusion-offload-config` from the Blackwell YAML to
  keep the model resident in unified memory and avoid offload transfers.

##### FastH3 Omni on eight GPUs

Install the attention kernels, then start the server:

```bash
uv pip install -e '.[cudnn-bsa]'

VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=7200 \
vllm serve FastVideo/FastVideo-FastH3-Omni-8-Step-V1 --omni \
  --config recipes/FastVideo/fasth3-8step-8gpu.yaml
```

Wait for `Application startup complete`, then use the image-and-prompt request
above. The configuration sets eight inference steps and handles GPU placement.

For an NVFP4 video decoder on Blackwell, add this to the deployment YAML:

```yaml
additional-config:
  video_vae_decoder:
    model: FastVideo/FastH3-NVFP4-VAE-Decoder
    subfolder: fast-8L
```

Use `quality-36L` for the deeper decoder, then restart the server.

**Performance reference: eight RTX PRO 5000 Blackwell 72 GB GPUs.**
BF16, 1344×768, one request at a time; timing includes MP4 encoding and
excludes model loading:

| Output | Generation time | Reported allocator peak |
|---|---:|---:|
| 124 frames, about 5 seconds | 41.04 s, median of 3 runs | 35.15 GiB |
| 345 frames, about 14 seconds | 95.93 s, 1 run | 38.94 GiB |

These are offline measurements. See the
[benchmark details](https://github.com/lishunyang12/vllm-omni/pull/63)
for the measured environment and configuration.

##### FastH3 V2: text to video and audio

This eight-GPU deployment is experimental. Install the `cudnn-bsa` extra
above, then start V2 in place of the Omni server:

```bash
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=7200 \
vllm serve FastVideo/FastVideo-FastH3-8-Step-V2 --omni \
  --config recipes/FastVideo/fasth3-8step-8gpu.yaml
```

Wait for startup, then send a text-only request in a second terminal:

```bash
curl --fail-with-body -sS http://localhost:8000/v1/videos/sync \
  -F 'prompt=A train crosses a stone bridge at sunrise, with rhythmic wheels and distant birdsong.' \
  -F 'size=1344x768' -F 'seconds=5' \
  -o fasth3-v2.mp4
```

Keep the model's default eight steps and sampling settings.

#### FastH3 four-step Dense and VSA adapters

Use the four-step adapter for text-to-video generation on eight GPUs.
Choose the adapter at startup; per-request switching and CPU/layer offload
are unsupported. Download the Dense and VSA adapters:

```bash
hf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
  dense-datafree/adapter_model.safetensors \
  vsa-datafree/adapter_model.safetensors \
  --local-dir ./FastH3-LoRA
```

Start with the Dense adapter:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve MiniMaxAI/MiniMax-H3 --omni \
  --host 127.0.0.1 --port 8095 --trust-remote-code \
  --task-type fl2va --served-model-name FastH3 \
  --num-gpus 8 --tensor-parallel-size 1 --usp 8 --ring 1 \
  --vae-patch-parallel-size 8 --vae-parallel-mode tile --vae-use-tiling \
  --diffusion-attention-config.default.backend FLASH_ATTN \
  --lora-path ./FastH3-LoRA/dense-datafree/adapter_model.safetensors
```

For the VSA adapter, install `uv pip install -e '.[vsa]'`, change `--lora-path`
to `./FastH3-LoRA/vsa-datafree/adapter_model.safetensors`, and replace the
attention option with:

```bash
--diffusion-attention-config.default.backend FASTVIDEO_VSA \
--diffusion-attention-config.default.fastvideo_vsa_topk 64
```

Keep pure Ulysses parallelism (`--usp 8 --ring 1`) for VSA. See
[backend selection](https://github.com/lishunyang12/vllm-omni/blob/pr/fasth3-omni-upstream/recipes/FastVideo/FastH3.md#hardware-and-backend-selection)
for hardware requirements.

After the server starts, generate a video in a second terminal:

```bash
curl --fail-with-body -sS http://127.0.0.1:8095/v1/videos/sync \
  -F 'model=FastH3' \
  -F 'prompt=A cinematic starship glides through a luminous nebula while its engines hum and distant radio chatter fills the cabin.' \
  -F 'width=1344' -F 'height=768' -F 'aspect_ratio=16:9' -F 'fps=24' \
  -F 'num_inference_steps=4' -F 'guidance_scale=1' -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":10.0}' \
  -o fasth3.mp4
```

Use four steps and `task=t2va`; keep the default sampling settings.

**Performance reference: eight B300 GPUs.** At 1344×768 and 24 FPS,
server generation time including MP4 encoding was:

| Output duration | Dense (TRTLLM) | VSA |
|---|---:|---:|
| 10 seconds | 9.84 s | 7.28 s |
| 15 seconds | 14.20 s | 10.80 s |

See [benchmark details](https://github.com/vllm-project/vllm-omni/pull/6909)
for the measured environment and configuration.

#### LightX2V Turbo few-step LoRA

[LightX2V Turbo](https://huggingface.co/lightx2v/Minimax-h3-Turbo) is a family of
few-step LoRA artifacts for H3. It reaches the same goal as FastH3 by the opposite
route: instead of fusing full-rank deltas at startup, Turbo stays a standard LoRA that
each request activates by name. One server can therefore hold several artifacts and let
requests choose between them, at the cost of the LoRA matmuls on every denoise step.

Eight Diffusers-layout artifacts are supported. The filename is the contract — the server
parses the task family, step count and resolution out of it, so a renamed file is rejected
rather than served on a guess. `audio_flow_shift` is `3.0` across the family.

| Artifact | Tasks | `num_inference_steps` | `flow_shift` | alpha |
|---|---|---:|---:|---:|
| `minimax_h3_fl2v_turbo_4step_v0.1.safetensors` | T2VA / FL2VA | 5 | 12 | none → 8 |
| `minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors` | T2VA / FL2VA | 5 | 6 | 128 |
| `minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors` | T2VA / FL2VA | 5 | 6 | 128 |
| `minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors` | T2VA / FL2VA | 5 | 6 | 8 |
| `minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors` | T2VA / FL2VA | 9 | 12 | 8 |
| `minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors` | T2VA / FL2VA | 9 | 6 | 8 |
| `minimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensors` | Ref2VA | 5 | 12 | 8 |
| `minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors` | Ref2VA | 9 | 6 | 8 |

Requested sigma points are always one more than the artifact's forwards, which is why a
four-step Turbo artifact takes `num_inference_steps=5` where FastH3 takes `4`.

Every artifact is rank 128 and the delta is applied at `scale * alpha / 128`, with alpha
read from the artifact's metadata — only `4step_v0.1` declares none and falls back to 8
with a warning.

Download one artifact under the H3 root so the existing bind mount includes it. Each
artifact except `4step_v0.1` also ships a `_comfyui_` export that fuses Q/K/V into a single
projection; those are **not** supported and are refused by name.

```bash
hf download lightx2v/Minimax-h3-Turbo \
  minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors \
  --local-dir /path/to/MiniMax-H3/Turbo-LoRA
```

Serve it on the same eight-GPU pure-Ulysses topology. `--lora-path` preloads the adapter;
it accepts one artifact, or a directory holding exactly one — a directory with several
recognized artifacts is rejected as ambiguous.

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 127.0.0.1 \
  --port 8095 \
  --trust-remote-code \
  --task-type fl2va \
  --served-model-name MiniMaxAI/MiniMax-H3 \
  --num-gpus 8 \
  --usp 8 \
  --ring 1 \
  --vae-patch-parallel-size 8 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend TRTLLM_ATTN \
  --lora-backend peft \
  --lora-path /path/to/MiniMax-H3/Turbo-LoRA/minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors
```

Unlike FastH3, the request must activate the adapter and carry that row's sampling
contract. A request whose steps or shift do not match the loaded artifact is rejected, so
a mismatch cannot silently sample the distilled model at noise levels it never saw.

```bash
curl --fail-with-body -sS -X POST http://127.0.0.1:8095/v1/videos/sync \
  -F 'prompt=A cinematic starship glides through a luminous nebula while its engines hum and distant radio chatter fills the cabin.' \
  -F 'width=1344' \
  -F 'height=768' \
  -F 'aspect_ratio=16:9' \
  -F 'fps=24' \
  -F 'num_inference_steps=5' \
  -F 'flow_shift=6' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":10.0,"audio_flow_shift":3.0}' \
  -F 'lora={"name":"h3-turbo-v1.2","path":"/path/to/MiniMax-H3/Turbo-LoRA/minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors","scale":1.0}' \
  -o turbo.mp4
```

FL2VA artifacts serve `t2va` and `fl2va` on an FL2VA or combined server. The two `ref2v`
rows need `--task-type ref2va`: a combined server answers `ref2va` from a second DiT the
adapter cannot bind to, so loading one there is refused rather than running an undistilled
model on a few-step ladder. The integration is dynamic-only — no prefusion, no LoRA
composition. DLO works by keeping the request-switchable LoRA A/B buffers resident on the
accelerator while DLO streams the base blocks; model-level and standard layerwise offload
remain unsupported.

Measured on 8×B300 with the command above, at 1344×768, 24 FPS, a 15 s FL2VA request with
first and last keyframes (362 frames), fixed seed, one discarded warmup and three recorded
runs. The baseline is the same server without an adapter at `num_inference_steps=50` and
`flow_shift=12`:

| Metric | Base H3, 49 forwards | Turbo v1.2, 4 forwards | Speedup |
|---|---:|---:|---:|
| Denoise loop | ~105 s | ~9 s | 11.7× |
| Per denoise step | 2.14–2.16 s/it | 2.20–2.42 s/it | — |
| Diffusion engine | 111.1–111.3 s | 15.0–15.1 s | 7.4× |
| Server E2E incl. MP4 | 112.5–112.9 s | ~16.5 s | 6.8× |

The ~10% higher per-step cost is the LoRA matmul the fused FastH3 route does not pay —
irrelevant against a 12× cut in forwards, but the reason a single-artifact deployment can
still prefer a fused student.

These figures are for a server serving Turbo requests back to back. Alternating between
Turbo and adapter-free requests on one server additionally pays a LoRA rebind on each
switch — measured at about 24 s per switch in the same run (39.2 s diffusion engine against
15.1 s steady state), because every one of H3's 208 LoRA layers is torn down and re-uploaded.
Give Turbo its own process if the workload mixes, or track vLLM-Omni #7195, which removes
that rebind.

#### RTX PRO 5000 Blackwell — four PCIe GPUs

Select the `RTX PRO 5000 Blackwell 4x` hardware profile for the validated BF16,
no-offload topology: TP2 × Ulysses2, Ring1, text-encoder TP4, VAE patch parallel 4
in tile mode, and `CUDNN_ATTN`. On the measured dual-socket host,
`CUDA_VISIBLE_DEVICES=0,2,1,3` kept the TP pairs on local PXB paths; check
`nvidia-smi topo -m` and adapt the order on other systems.

This 72 GiB/GPU profile serves the root path but selects one DiT with `--task-type`.
The task picker adds `fl2va` for T2VA/FL2VA and `ref2va` for Ref2VA; loading both DiTs
together is not qualified on this memory-constrained profile.

A five-step topology screen after warmup produced decodable H.264 video with stereo AAC:

| Workload | Client E2E | Encode | Denoise | VAE decode | Peak/GPU |
|---|---:|---:|---:|---:|---:|
| T2VA, 124 frames, 1344×768 | 32.993 s | 0.034 s | 27.915 s | 4.307 s | 67.6 GiB |
| FL2VA first frame, same output shape | 30.130 s | 0.276 s | 24.396 s | 4.339 s | 67.6 GiB |

These are topology-screening results, not 50-step throughput claims. Nsight Systems
aggregation for the same run was:

| Workload | AllGather | Send/Recv | NCCL other | NCCL total | Dense FMHA | max-min/mean |
|---|---:|---:|---:|---:|---:|---:|
| T2VA | 0.93% | 8.53% | 14.63% | 24.09% | 31.15% | 1.11% |
| FL2VA | 0.89% | 7.26% | 15.76% | 23.91% | 32.53% | 0.13% |

`NCCL other` covers NCCL kernels outside recognized AllGather and point-to-point
Send/Recv, including AllReduce, ReduceScatter, or Broadcast when present.

#### AMD ROCm — four MI300X GPUs

Verified on 4× AMD Instinct MI300X (gfx942) with the ROCm H3 image, AITER packed
variable-length attention, Ulysses 4, text-encoder TP4, and native tiled VAE patch
parallelism 4:

```bash
docker pull vllm/vllm-omni-rocm:minimax-h3

HIP_VISIBLE_DEVICES=0,1,2,3 \
VLLM_ROCM_USE_AITER=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend FLASH_ATTN
```

On ROCm, `FLASH_ATTN` resolves to AITER on supported Instinct architectures. Do not
install the CUDA-only `[fa4]` extra or set `FLASHINFER_DISABLE_VERSION_CHECK`.

The refreshed `vllm/vllm-omni-rocm:minimax-h3` tag includes the Ref2VA soundfile
fallback, TorchCodec, and FFmpeg, so T2VA, FL2VA, and the complete Ref2VA path use the
same official image.

#### AMD ROCm — four MI355X GPUs

Verified on 4× AMD Instinct MI355X. The official ROCm image includes the
MiniMax H3 Ref2VA fixes, TorchCodec, and FFmpeg. We tested both Ulysses 4 with Ring 1
and Ulysses 1 with Ring 4. The command below shows the Ulysses 4 / Ring 1 configuration
with tiled VAE patch parallelism degree 4, AITER, and CPU offload:
```bash
docker pull vllm/vllm-omni-rocm:minimax-h3

HIP_VISIBLE_DEVICES=0,1,2,3 \
VLLM_ROCM_USE_AITER=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --enable-cpu-offload \
  --text-encoder-tp-size 4
```

This root-model command accepts T2VA, FL2VA, and Ref2VA requests without restarting.

Three constraints on this configuration:

- **Do not add `--enforce-eager`.** Regional compile is the default
  (`--diffusion-compile-granularity regional`) and is worth ~9.5%. The first request
  includes compilation — warm the server once before measuring steady state.
- **`--cfg-parallel-size` must stay 1.** H3 is CFG-distilled and has no negative branch;
  anything greater is rejected with an explicit error rather than silently duplicating a
  branch that does not exist.
- **The VAE supports its native `tile` mode only** — not `spatial_shard_width` or
  `spatial_shard_height` (those belong to the distributed Wan autoencoder). Patch
  parallel size must be 1 or the full DiT group size.

### Launch — single GPU (accuracy and memory first)

Model-level CPU offload keeps the Qwen3-VL encoder and the DiT from being resident
simultaneously. This is the accuracy-qualified reference path; it trades PCIe/NVLink
transfer latency for a much smaller GPU footprint, and needs enough system RAM for the
offloaded components.

```bash
CUDA_VISIBLE_DEVICES=0 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --task-type fl2va \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 1 \
  --enable-cpu-offload \
  --diffusion-attention-backend FLASH_ATTN
```

### Launch — DGX Spark (GB10, 128 GB unified memory)

GB10 is a **unified-memory** platform: the 128 GB LPDDR5X (≈121 GiB usable) is shared by
CPU and GPU, so both offload strategies used elsewhere in this recipe stop being capacity
levers. `--enable-cpu-offload` and distributed layerwise offload move weights from "VRAM"
to "host RAM", which is the same physical pool here — nothing is freed. DLO is actively
harmful: staging rank-local shards into pinned host memory creates a second copy
alongside the resident one, and the process is killed by the OOM killer (`Exit code: -9`)
right after `Enabling offloader backend`.

One partition on disk is 135 GiB in BF16:

| Component | Size | Quantized by `--quantization fp8` |
|---|---:|---|
| `text_encoder` (Qwen3-VL) | 63 GiB | no |
| `transformer` (DiT) | 62 GiB | yes |
| `video_vae` | 9.8 GiB | no |
| `audio_vae` | 0.6 GiB | no |

135 GiB does not fit in 121 GiB, so **FP8 is mandatory** and only one DiT can be
resident — always pass `--task-type fl2va` or `--task-type ref2va`. Online FP8 touches
the DiT only, bringing the partition to roughly 104 GiB before activations.

Weight quantization runs at load time and is slow on this platform; `--init-timeout 3600`
keeps startup from being cut short.

```bash
CUDA_VISIBLE_DEVICES=0 \
FLASHINFER_DISABLE_VERSION_CHECK=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=7200 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --task-type fl2va \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --init-timeout 3600 \
  --num-gpus 1 \
  --usp 1 \
  --ring 1 \
  --quantization fp8 \
  --vae-patch-parallel-size 1 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --enforce-eager \
  --diffusion-attention-backend CUDNN_ATTN
```

Restart with `--task-type ref2va` to serve the Ref2VA family. Keep requests at a 576 px
short edge (1024×576) and 5 s duration as the starting shape; GB10 has one GPU and no
parallelism headroom, so wall-clock per request is far higher than the multi-GPU profiles
above.

### Launch — single Ascend 950PR NPU (INT8)

vLLM-Omni serves H3 on Ascend through the `vllm-ascend` platform plugin, on a
CPU-only aarch64 PyTorch build with `torch_npu` — no CUDA runtime is present.
One 950PR holds a single DiT partition, so this profile serves the `FL2VA`
directory and uses online INT8 in place of the sharding levers the multi-GPU
profiles rely on.

Three Ascend-specific constraints come before the command:

- **Attention backend.** `CUDNN_ATTN`, `FLASHINFER_ATTN`, `TRTLLM_ATTN`, and the
SageAttention backends are CUDA-only; selecting one on NPU fails with a
`NotImplementedError` out of `forward_npu`. The NPU platform default resolves
to `FLASH_ATTN` when MindIE-SD is installed and `TORCH_SDPA` otherwise, and
`FLASH_ATTN` is where H3's NPU memory work lives: a mask-free packed varlen
path that never materializes the quadratic padding mask, plus K/V prefix
slicing. Install MindIE-SD *before* passing
`--diffusion-attention-backend FLASH_ATTN` — the backend imports `mindiesd`
lazily and otherwise raises `ImportError` at the first denoise step. Once
installed the platform default picks it up on its own.
- **Shared memory.** Decoded video crosses the diffusion-worker process boundary
through POSIX shared memory — roughly 0.9 GiB for a 5 s 1024×576 clip. The
64 MiB Docker default is large enough for the control-plane queues, so the
server starts and the denoise loop reaches 100% before the write into
`/dev/shm` raises SIGBUS and kills the worker with `Bus error (core dumped)`.
Check `df -h /dev/shm` and start the container with `--shm-size=8g` or
`--ipc=host`. It cannot be changed on a running container, and an in-container
`mount -o remount` needs `CAP_SYS_ADMIN`.
- **No `--enforce-eager`.** It disables regional compilation and only slows the
steady state; the first request is a warmup either way. `--cfg-parallel-size`
must stay 1, as everywhere else in this recipe.

```bash
ASCEND_RT_VISIBLE_DEVICES=0 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3/FL2VA \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 1 \
  --tensor-parallel-size 1 \
  --usp 1 \
  --ring 1 \
  --text-encoder-tp-size 1 \
  --vae-patch-parallel-size 1 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --init-timeout 14400 \
  --stage-init-timeout 14400 \
  --quantization int8
```

### Text-encoder tensor parallelism

By default the whole Qwen3-VL encoder is resident on the DiT main rank, making it the
peak-memory hotspot on multi-GPU no-offload runs. `--text-encoder-tp-size N` shards it
across the first `N` DiT ranks using vLLM-style tensor-parallel layers on a dedicated
encoder process group:

```bash
vllm serve /path/to/MiniMax-H3 \
  --omni --trust-remote-code \
  --task-type fl2va \
  --num-gpus 4 --usp 4 --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 --vae-parallel-mode tile --vae-use-tiling
```

- `N` must divide the Qwen3-VL head counts (64 attention heads / 8 KV heads) → 1, 2, 4 on
  a 4-GPU server; 1, 2, 4, 8 on 8 GPUs.
- On 4 GPUs, `N=4` drops the per-rank engine peak from **133 GB to 103 GB** (~35.7 GB) at
  no measurable throughput cost, while each non-main rank gains ~51.5/N GB.
- Numerically: `N=1` reproduces the HF reference path bit-exactly (`max_abs = 0`); `N=4`
  introduces bounded BF16 rounding only (row-parallel all-reduce in FP32), giving
  31.11 dB PSNR / 0.9566 SSIM end-to-end against `N=1` with identical structure.
- Combining `N>1` with regional compile used to produce a `ValueError: v must be finite`
  on the *second* request. That is fixed — the token refiner's attention runs on
  replicated rows before `sp_prepare` and was wrongly getting a Ulysses SP all-to-all.

### Key request parameters

| Parameter | Recommended | Notes |
|---|---|---|
| `task` | `t2va`, `fl2va`, `ref2va` | Passed inside `extra_params`; must be compatible with `--task-type` when one is set |
| `duration` | 4–15 s | Decimal seconds in `extra_params`; snapped to H3's legal `17n+5` frame count (15 s → 362 frames / 15.083 s) |
| `fps` | `24` | Output FPS is fixed |
| `num_inference_steps` | `50` | Matches the reference accuracy workloads |
| `flow_shift` | `12` | Video sigma shift |
| `audio_flow_shift` | `3` | Audio sigma shift, in `extra_params` |
| `width`, `height` | see below | Server-side validation: multiples of 32, aspect ratio 1:4 to 4:1 |
| `seed` | fixed value | Output is deterministic per request at a fixed seed |

On `width`/`height`, the serving-side check is broader than the documented product modes —
pick a **short edge of 768 or 1440** in one of the supported ratios (21:9, 16:9, 4:3, 1:1,
3:4, 9:16) to stay on shapes the model was trained and validated for. FL2VA can omit both
and inherit the first frame's aspect ratio at a 768 px short edge. Prompts cap at 7,000
characters.

Use `POST /v1/videos/sync` when you want the MP4 in the response body; `POST /v1/videos`
is the async job-polling variant.

### Validated performance (4×B300)

These measurements used task-specific single-DiT processes before the modular
combined-service merge; they remain the validated per-task operating points.

| Workload | Config | Result |
|---|---|---|
| FL2VA, 209 frames, 1248×768 (8.7 s) | no offload, U4, VPP4 tile, regional compile | **86.96 s** client E2E (±0.23 s) |
| Two-video Ref2VA, 362 frames, 1344×768 (15 s) | same | **784.4 s** accounted model-stage mean |

Stage split for the FL2VA case: text encoder 0.21 s, visual encoder 0.20 s, DiT 79.1 s
(**88%** of the request), video+audio VAE decode 2.40 s. VAE patch parallelism is the
cheapest win available — it cuts decode 3.4–3.5× (8.24 s → 2.40 s).

#### Validated performance (4×MI300X)

Measured with `vllm/vllm-omni-rocm:minimax-h3`
(`sha256:29d1946af9c69e3e0a7128c247bdc8c82437ead43fbc366fed443533cf6ce9e8`),
BF16, AITER, U4, text-encoder TP4, VPP4 tile, regional compile, one warmup, and
the synchronous Video API:

| Workload | Model stages | Client E2E |
|---|---:|---:|
| T2VA, 209 frames, 1344×768 (8.7 s) | encode 0.09 s, denoise 244.04 s, decode 4.15 s | **267.42 s** |
| FL2VA, 209 frames, 1344×768 (8.7 s) | encode 13.98 s, denoise 257.58 s, decode 4.11 s | **287.07 s** |

Both outputs were validated as 209-frame H.264 at 24 FPS with 32 kHz stereo AAC.
A v0.26.0 source build additionally completed image+audio and video-reference Ref2VA,
plus a 2048×1088 T2VA run. These measurements describe tested shapes, not a general
latency guarantee.

A `with_stack` profile of the two-video Ref2VA case shows the pipeline is
**GPU-attention-bound, not host-starved**: FlashAttention-4 is ~76% of `diffuse` device
time, Ulysses send/recv is secondary, GPU utilization is 92.8–94.3% and the CPU-idle
union inside the transformer forwards is 3.15%. Reducing DiT time further needs a change
in attention cost, which is not lossless;
[SAGE + Skip-Softmax](#optional-lossy-attention-sage--skip-softmax) above is the supported
way to trade attention precision and sparsity for speed on datacenter Blackwell.

Against the checkpoint's own reference implementation, the accuracy-qualified path scores
SSIM 0.9873–0.9896, PSNR 39–42 dB, pixel cosine 0.9996+, and audio log-mel cosine
0.977–0.996 on T2VA/FL2VA. The two-video Ref2VA composite scores lower on raw pixels
(SSIM 0.628) largely because of H.264 bitrate differences, while CLIP cosine is 0.9816 and
audio log-mel cosine 0.9869 — content and motion stay aligned.

#### Cache-DiT

Opt-in only. A 50-step T2VA probe cut denoise time **30.2%** (121.01 s → 84.47 s) but
dropped SSIM to **0.831** against the uncached output. H3's video trajectory is sensitive
to cached denoise steps — choose the cache budget against your own quality target.

#### Offload and accuracy

On a **single GPU**, model-level CPU offload is the accuracy-qualified path. On four GPUs,
no-offload is both faster and accuracy-clean once the encoder stays resident: an earlier
no-offload build performed a ~63 GiB DiT GPU→CPU→GPU round trip inside `encode_prompt`,
which was pure overhead (encoder 30.38 s → 0.207 s once removed, ~146×).

### Known limitations

- The modular root service loads both DiTs by default. Capacity profiles use
  `--task-type fl2va` or `--task-type ref2va` and expose only that task family.
- H3 executes **one generation request per diffusion batch** today.
- The first request is a compile warmup only on the resident regional-compile profiles.
  The two-GPU DLO recipes use `--enforce-eager` and do not compile the DiT.
- **The serving path accepts fewer references than the model supports.** H3 documents up
  to 9 images, 3 video clips, and 3 audio clips (12 files) per Omni Reference request;
  the current vLLM-Omni path takes exactly one image plus one audio reference, **or** one
  or more videos with **no** separate `audio_reference` (it uses the source soundtracks).
- The 768 px short-edge mode is available for T2VA and FL2VA; 1344×768 is the documented
  16:9 request shape. The 24 GB DLO recipe uses 1024×576 as its lower-memory starting
  point.
- `--cfg-parallel-size > 1` is rejected by design (CFG-distilled, no negative branch).
- The VAE supports the native `tile` parallel mode only.
- A `U2 × Ring2` hybrid currently fails with an attention-mask length mismatch; use pure
  Ulysses.
- FastH3 preview v1 serves T2VA only, is fused at startup, and cannot use model-level,
  layerwise, or distributed layerwise offload. Its VSA variants additionally require CUDA,
  `fastvideo-kernel`, `FASTVIDEO_VSA`, and pure local/Ulysses attention; Ring and AllGather
  sequence parallelism are rejected.
- Online FP8 quantizes the DiT only — the Qwen3-VL text encoder and both VAEs stay
  BF16. It is incompatible with layerwise offload, whose weight stride is rejected by
  the Cutlass FP8 kernel.
- On unified-memory platforms (DGX Spark / GB10) CPU offload and DLO give no capacity
  benefit, because host RAM and VRAM are one pool; DLO additionally doubles peak usage
  while staging. Use resident FP8 with a single `--task-type` partition instead.
- Pure Ulysses still replicates the 66.3 GB DiT on every rank, so 64 GB GPUs cannot use
  `--usp N --tp 1` as a resident capacity path. Use DiT TP plus DLO (the two-GPU recipes
  above), or model-level CPU offload; text-encoder TP alone is not sufficient.
- DLO is a capacity path, not a free speedup: it needs substantial host RAM and streams
  weights over PCIe on every denoise step. Increase `--dlo-resident-layers` only after
  checking peak HBM on the exact request shape.

### References

- [Model card](https://huggingface.co/MiniMaxAI/MiniMax-H3)
- [Modular FL2VA/Ref2VA pipeline (vLLM-Omni #5720)](https://github.com/vllm-project/vllm-omni/pull/5720)
- [vLLM-Omni Video API](https://docs.vllm.ai/projects/vllm-omni/en/latest/serving/videos_api/)
- [vLLM-Omni diffusion parallelism](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/parallelism/overview/)
- [vLLM-Omni diffusion attention backends](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/attention_backends/)
- [FastVideo VSA attention guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/attention_backends/fastvideo_vsa/)
- [FastH3 preview adapters](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA)
- [FastH3 family and deployment guides](https://github.com/lishunyang12/vllm-omni/blob/pr/fasth3-omni-upstream/recipes/FastVideo/README.md)
- [FastH3 Omni and V2 integration (vLLM-Omni #9167)](https://github.com/vllm-project/vllm-omni/pull/9167)
- [MiniMax-H3 VSA and Ulysses support (vLLM-Omni #6909)](https://github.com/vllm-project/vllm-omni/pull/6909)
- [vLLM-Omni TRTLLM attention guide (SAGE, Skip-Softmax)](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/attention_backends/trtllm/)
- [MiniMax-H3 production serving on vLLM-Omni (blog)](https://vllm.ai/blog/2026-09-01-minimax-h3-production-serving)
- [vLLM-Omni supported models](https://docs.vllm.ai/projects/vllm-omni/en/latest/models/supported_models/)
- [vLLM-Omni GPU installation](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/gpu/)
- [ROCm H3 tracking issue](https://github.com/vllm-project/vllm-omni/issues/5697)