{ "hf_id": "black-forest-labs/flux-3-image-klein", "meta": { "title": "FLUX 3 Image Klein", "slug": "flux-3-image-klein", "provider": "Black Forest Labs", "description": "5.6B distilled image transformer for text-to-image and image editing, served via vLLM-Omni with a Qwen3-VL-4B text encoder", "date_added": "2026-10-15", "date_updated": "2026-10-15", "tasks": [ "omni" ], "performance_headline": "1024x1024 in ~2s on a single GB200 (16-step distilled schedule)", "related_recipes": [], "hardware": { "gb200": "verified" } }, "model": { "model_id": "black-forest-labs/flux-3-image-klein", "min_vllm_version": "0.32.0", "docker_image": "vllm/vllm-omni:flux-3-image-klein", "architecture": "dense", "parameter_count": "5.6B", "active_parameters": "5.6B", "context_length": 0, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git", "note": "installs vLLM-Omni from source (main) — FLUX 3 Image Klein support ships in the upcoming release" }, "docker": { "command": "docker pull vllm/vllm-omni:flux-3-image-klein", "note": "Recommended: the day-0 image ships vLLM-Omni with FLUX 3 Image Klein support" }, "extras": [ { "note": "Accept the FLUX Kommunity License on Hugging Face first, then authenticate.", "command": "hf auth login" } ] } }, "features": {}, "opt_in_features": [], "variants": { "default": { "label": "BF16", "precision": "bf16", "vram_minimum_gb": 24, "description": "Root BF16 distilled package (5.6B DiT + Qwen3-VL-4B text encoder + VAE), fixed 16-step schedule, no guidance", "supported_hardware": [ "h100", "h200", "b200", "gb200", "b300", "gb300", "rtx_pro_6000" ] }, "fp8r": { "label": "FP8R", "precision": "fp8", "precision_hardware": { "brand": "NVIDIA" }, "vram_minimum_gb": 17, "description": "FP8R DiT variant from the package's variants/fp8r folder; text encoder and VAE stay BF16", "supported_hardware": [ "h100", "h200", "b200", "gb200", "b300", "gb300" ], "extra_args": [ "--stage-overrides", "{\"0\":{\"additional_config\":{\"variant\":\"fp8r\"}}}" ] }, "nvfp4": { "label": "NVFP4", "precision": "nvfp4", "precision_hardware": { "brand": "NVIDIA" }, "vram_minimum_gb": 13, "description": "NVFP4 DiT variant from the package's variants/nvfp4 folder; text encoder and VAE stay BF16", "supported_hardware": [ "b200", "gb200", "b300", "gb300" ], "extra_args": [ "--stage-overrides", "{\"0\":{\"additional_config\":{\"variant\":\"nvfp4\"}}}" ] } }, "hardware_overrides": {}, "omni": { "port": 8099, "variant_label": "DiT Variant", "variant_hint": "Choose BF16 for the root package or FP8R/NVFP4 to shrink DiT memory. Variants live in the checkpoint's variants/ folder.", "task_label": "Generation Mode", "task_hint": "T2I hits /v1/images/generations; I2I sends the reference image through /v1/chat/completions.", "task_axes": [ { "id": "x2i", "label": "X2I", "options": [ { "id": "t2i", "label": "T2I" }, { "id": "i2i", "label": "I2I" } ] } ], "tasks": [ { "id": "t2i", "key": "t2i", "label": "Text-to-image", "axes": { "x2i": "t2i" }, "vram_minimum_gb": 24, "endpoint": "/v1/images/generations", "description": "Text-to-image generation with the package default 1024x1024 size and 16-step distilled schedule.", "curl": "curl -sS --fail-with-body \\\n -X POST http://localhost:8099/v1/images/generations \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"flux-3-image-klein\",\n \"prompt\": \"a serene alpine meadow at golden hour, wildflowers in the foreground\",\n \"size\": \"1024x1024\",\n \"n\": 1,\n \"response_format\": \"b64_json\",\n \"seed\": 42\n }' | python3 -c 'import base64,json,sys; open(\"t2i.png\",\"wb\").write(base64.b64decode(json.load(sys.stdin)[\"data\"][0][\"b64_json\"]))'\n" }, { "id": "i2i", "key": "i2i", "label": "Image-to-image", "axes": { "x2i": "i2i" }, "vram_minimum_gb": 24, "endpoint": "/v1/chat/completions", "description": "Reference-based editing / restyling, with up to 10 reference images. Each image rides as a base64 data URI in a chat message.", "curl": "export REF_IMAGE=/absolute/path/to/reference.png\n\nDATA_URI=\"data:image/png;base64,$(base64 -w0 \"$REF_IMAGE\")\"\ncurl -sS --fail-with-body \\\n -X POST http://localhost:8099/v1/chat/completions \\\n -H 'Content-Type: application/json' \\\n -d \"{\n \\\"model\\\": \\\"flux-3-image-klein\\\",\n \\\"messages\\\": [{\n \\\"role\\\": \\\"user\\\",\n \\\"content\\\": [\n {\\\"type\\\": \\\"text\\\", \\\"text\\\": \\\"restyle as a watercolor painting with soft pastel tones\\\"},\n {\\\"type\\\": \\\"image_url\\\", \\\"image_url\\\": {\\\"url\\\": \\\"${DATA_URI}\\\"}}\n ]\n }],\n \\\"seed\\\": 42,\n \\\"height\\\": 1024,\n \\\"width\\\": 1024\n }\" | python3 -c 'import base64,json,sys; u=json.load(sys.stdin)[\"choices\"][0][\"message\"][\"content\"][0][\"image_url\"][\"url\"]; open(\"i2i.png\",\"wb\").write(base64.b64decode(u.split(\",\",1)[1]))'\n" } ] }, "guide": "## Overview\n\n[FLUX 3 Image Klein](https://huggingface.co/black-forest-labs/flux-3-image-klein)\nis a 5.6B distilled image transformer from Black Forest Labs for text-to-image\ngeneration and image editing (single- and multi-reference). It uses\nQwen3-VL-4B-Instruct as the text encoder and runs a fixed 16-step schedule\nwith no classifier-free guidance. vLLM-Omni serves it through an\nOpenAI-compatible API:\n\n| Mode | Endpoint | Request body |\n|------|----------|--------------|\n| T2I | `/v1/images/generations` | JSON (`prompt`, `size`, `seed`) |\n| I2I | `/v1/chat/completions` | Chat message with the reference image as a base64 data URI |\n\nThe checkpoint is a single package: DiT (`model.safetensors`), image\nautoencoder (`ae.safetensors`), and `text_encoder/` at the root, with\nquantized DiT packages under `variants/fp8r` and `variants/nvfp4`. Select a\nvariant with\n`--stage-overrides '{\"0\":{\"additional_config\":{\"variant\":\"fp8r\"}}}'`; omit it\nfor the BF16 root package.\n\n## Prerequisites\n\n- Accept the [FLUX Kommunity License](https://huggingface.co/black-forest-labs/flux-3-image-klein/blob/main/LICENSE.md) on Hugging Face, then `hf auth login`. The license permits non-commercial use with a commercial-output exception for Qualifying Users.\n- Recommended: use the day-0 Docker image `vllm/vllm-omni:flux-3-image-klein`, which ships vLLM-Omni and all dependencies. Bare-metal installs use vLLM-Omni from source (see Install above).\n\n## Launching the server\n\nWith the day-0 image:\n\n```bash\ndocker run --runtime nvidia --gpus all \\\n -v ${HF_HOME:-$HOME/.cache/huggingface}:/root/.cache/huggingface \\\n --env \"HF_TOKEN=$HF_TOKEN\" \\\n -p 8099:8099 \\\n --ipc=host \\\n vllm/vllm-omni:flux-3-image-klein \\\n vllm serve black-forest-labs/flux-3-image-klein --omni \\\n --served-model-name flux-3-image-klein \\\n --port 8099\n```\n\nBare metal:\n\n```bash\nvllm serve black-forest-labs/flux-3-image-klein --omni \\\n --served-model-name flux-3-image-klein \\\n --port 8099\n\n# FP8R / NVFP4 DiT variant\nvllm serve black-forest-labs/flux-3-image-klein --omni \\\n --served-model-name flux-3-image-klein \\\n --stage-overrides '{\"0\":{\"additional_config\":{\"variant\":\"fp8r\"}}}' \\\n --port 8099\n```\n\nUseful extras (see `examples/online_serving/flux3/run_server.sh` in the vLLM-Omni repository):\n\n- `--tensor-parallel-size N` / `--cfg-parallel-size N` for multi-GPU scaling.\n- `--fp8-attention` (FlashInfer/cuDNN FP8 DiT attention, SM100 only, needs cuDNN >= 9.17.1).\n- `--lora-path /path/to/adapter --lora-scale 1.0` to preload a PEFT LoRA adapter; each request still selects the adapter by path.\n- `uv pip install -e '.[prims-ts]'` enables PrimsTS kernels for FP8R/NVFP4 packages on SM100/SM103/SM107; BF16 always uses vLLM kernels.\n\n## Offline inference\n\n```bash\n# T2I\npython examples/offline_inference/flux3/end2end.py \\\n --model black-forest-labs/flux-3-image-klein \\\n --prompt \"a serene alpine meadow at golden hour, wildflowers in the foreground\"\n\n# I2I (conditioning image; repeat --image for multi-reference, up to 10)\npython examples/offline_inference/flux3/end2end.py \\\n --model black-forest-labs/flux-3-image-klein \\\n --image /absolute/path/to/reference.png \\\n --prompt \"restyle as a watercolor painting with soft pastel tones\" \\\n --seed 7\n```\n\n## Prompt upsampling (optional)\n\n[black-forest-labs/flux-3-image-klein-pup](https://huggingface.co/black-forest-labs/flux-3-image-klein-pup)\nis a Qwen3.5-9B vision-language fine-tune that expands a short request into\nthe detailed caption FLUX 3 Image Klein works best with — the reference\ninference code runs it in front of the image model by default. It is a\nstandard `Qwen3_5ForConditionalGeneration` checkpoint, so it is served with\nthe stock vLLM image on a separate GPU:\n\n```bash\nhf download black-forest-labs/flux-3-image-klein-pup --local-dir /path/to/pup\n\ndocker run --gpus '\"device=0\"' --ipc=host -p 8000:8000 \\\n -v /path/to/pup:/model:ro \\\n vllm/vllm-openai:v0.29.0 \\\n --model /model \\\n --served-model-name flux-3-image-klein-pup \\\n --max-model-len 32768 \\\n --gpu-memory-utilization 0.90 \\\n --reasoning-parser qwen3\n```\n\nThe system prompts (`system_prompts.json` in the pup repo) select\ntext-to-image vs. editing mode, and `t2i_bbox` / `i2i_bbox` variants return\nthe caption plus a JSON table of `[y_min, x_min, y_max, x_max]` boxes on the\n0-1000 grid. A ready-made client is `openai_client_pup.py` next to\n`openai_client.py` in `examples/online_serving/flux3/` of the vLLM-Omni\nrepository — pass `--image` repeatedly (up to 10) for multi-reference\nediting; feed its printed caption to the image model as the prompt.\n\n## Behavior and constraints\n\n- **Distilled package**: do not pass `guidance_scale` or a nonempty\n `negative_prompt` — the model rejects them. Step count and flow shift\n default to the package config (16 steps, shift 6.93); `--steps` and\n `--flow-shift` override them.\n- **Multi-reference editing**: I2I accepts up to 10 reference images, sent\n in order as multiple `image_url` entries in the chat message. Prompts can\n cite them by position (\"the first image\", \"the second image\", ...).\n- **I2I has no strength parameter**: the reference image is fed to the DiT as\n conditioning tokens and generation always starts from pure noise. The seed\n controls how far the output drifts from the reference — some seeds stay\n very close to the input, so change the seed when you want a bolder restyle.\n- Startup logs show two benign `Error parsing config ... Should have a\n 'model_type' key` lines: the package's root `config.json` is a custom FLUX\n format, and vLLM-Omni falls back to the Flux3 pipeline automatically.\n- Bounding boxes on a 0-1000 grid can be embedded in the prompt to place,\n move, recolor, or remove elements (see the model card).\n\n## Verified setup\n\nRecipe validated end-to-end on 2026-10-15 with the day-0 image build\n(arm64, CUDA 13.0) on a GB200: T2I 1024x1024 in ~2.1s and I2I in ~2.8s\nafter a ~105s engine startup; ~19 GiB GPU memory with the BF16 package.\n\n## References\n\n- [FLUX 3 Image Klein model card](https://huggingface.co/black-forest-labs/flux-3-image-klein)\n- [FLUX 3 Image Klein Base](https://huggingface.co/black-forest-labs/flux-3-image-klein-base) (undistilled, for fine-tuning / LoRA training)\n- [FLUX 3 Image Klein PUP](https://huggingface.co/black-forest-labs/flux-3-image-klein-pup) (prompt upsampler)\n- [vLLM-Omni](https://github.com/vllm-project/vllm-omni)\n", "source": { "source": "vllm-omni", "repo": "vllm-project/vllm-omni", "ref": "main", "sha": "cda4a1ef83b295ba332146a7709e6277725edc9f", "path": "recipes/models/black-forest-labs/flux-3-image-klein.yaml", "edit_url": "https://github.com/vllm-project/vllm-omni/edit/main/recipes/models/black-forest-labs/flux-3-image-klein.yaml", "overlays": [] } }