{
  "hf_id": "bosonai/higgs-audio-v3-tts-4b",
  "meta": {
    "title": "Higgs Audio V3 TTS",
    "slug": "higgs-audio-v3-tts-4b",
    "provider": "Boson AI",
    "description": "Boson AI's ~4B Qwen3-backbone text-to-speech model served via vLLM-Omni — 24 kHz speech, 100+ languages, zero-shot voice cloning with inline emotion/style/prosody control tokens — through the OpenAI /v1/audio/speech API.",
    "date_added": "2026-06-17",
    "date_updated": "2026-06-15",
    "tasks": [
      "omni"
    ],
    "related_recipes": [
      "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
      "fishaudio/s2-pro",
      "zai-org/GLM-TTS"
    ]
  },
  "model": {
    "model_id": "bosonai/higgs-audio-v3-tts-4b",
    "min_vllm_version": "0.22.0",
    "architecture": "dense",
    "parameter_count": "4B",
    "active_parameters": "4B",
    "context_length": 0,
    "base_args": [
      "--omni",
      "--trust-remote-code"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      },
      "extras": [
        {
          "note": "vLLM-Omni provides the TTS serving backend and the `--omni` serve path. See the installation guide for pre-built wheels or building from source.",
          "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git"
        }
      ]
    }
  },
  "omni": {
    "tasks": [
      {
        "id": "t2a",
        "label": "Text → Speech",
        "endpoint": "/v1/audio/speech",
        "description": "Higgs Audio V3 TTS via /v1/audio/speech (24 kHz mono)",
        "curl": "curl -X POST http://localhost:8000/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"bosonai/higgs-audio-v3-tts-4b\",\n    \"input\": \"Hello, how are you?\"\n  }' --output hello.wav\n"
      }
    ]
  },
  "features": {},
  "opt_in_features": [],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 24,
      "description": "BF16 weights — single GPU (reference run on 1x H100 80 GB; ~4B talker + codec decoder share the device)"
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\n[Higgs Audio V3 TTS](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b) is\nBoson AI's multilingual text-to-speech model served through **vLLM-Omni** with\nthe OpenAI-compatible `/v1/audio/speech` API. It generates **24 kHz** speech,\nsupports zero-shot voice cloning from a reference clip, and handles **100+\nlanguages** with inline control tokens for emotion, style, and prosody. The\narchitecture is a ~4B Qwen3 backbone with a fused multi-codebook embedding/head\n(8 codebooks × 1026 vocab, MusicGen-style delay pattern).\n\n## Prerequisites\n\n- **Hardware**: a single CUDA GPU. Reference run: 1x H100 80 GB — Stage 0\n  (talker, ~4B) ~60% GPU memory, Stage 1 (codec decoder) ~25%.\n- **vLLM-Omni** targeting vLLM >= 0.22.\n- `--trust-remote-code` is required.\n\n## Installation\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install git+https://github.com/vllm-project/vllm-omni.git\n```\n\n## Launch the server\n\nThe deploy config (`vllm_omni/deploy/higgs_multimodal_qwen3.yaml`) is\nauto-discovered from `model_type=higgs_multimodal_qwen3`:\n\n```bash\nvllm serve bosonai/higgs-audio-v3-tts-4b --omni --trust-remote-code \\\n  --host 0.0.0.0 --port 8000\n```\n\n## Client usage\n\n### Basic TTS (curl)\n\n```bash\ncurl -X POST http://localhost:8000/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"bosonai/higgs-audio-v3-tts-4b\", \"input\": \"Hello, how are you?\"}' \\\n  --output hello.wav\n```\n\n### Voice clone (Python client)\n\n```bash\npython examples/online_serving/text_to_speech/higgs_audio_v3/batch_speech_client.py \\\n  --base-url http://localhost:8000 \\\n  --model bosonai/higgs-audio-v3-tts-4b \\\n  --ref-audio path/to/reference.wav \\\n  --ref-text \"Transcript of the reference.\" \\\n  --prompts \"Text to clone.\"\n```\n\n### Offline batch inference\n\n```bash\npython examples/offline_inference/text_to_speech/higgs_audio_v3/end2end.py \\\n  --texts \"Hello world.\" \"The quick brown fox jumps over the lazy dog.\" \\\n  --output-dir results/higgs_v3_wavs\n```\n\n`ref_audio` accepts WAV/FLAC/MP3; `ref_text` is optional but improves fidelity.\n\n## Known limitations\n\n- Output is **24 kHz mono** WAV.\n- The deploy config runs both stages with `enforce_eager: true`\n  (`max_num_seqs=16`). A CUDA graph is available for Stage 0 and helps batch=1\n  (~-13% RTF) but not batch>1 (the model-owned sampler runs outside the graph);\n  Stage 1 (code2wav) must stay eager (`@torch.inference_mode` is incompatible\n  with graph capture).\n- Async (chunk-based) streaming is not yet implemented; the pipeline is\n  sync-only.\n\n## References\n\n- [Model card](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b)\n- [vLLM-Omni Higgs Audio V3 online example](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_speech/higgs_audio_v3)\n- [vLLM-Omni Higgs Audio V3 offline example](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/text_to_speech/higgs_audio_v3/end2end.py)\n",
  "source": {
    "source": "vllm-omni",
    "repo": "vllm-project/vllm-omni",
    "ref": "main",
    "sha": "cda4a1ef83b295ba332146a7709e6277725edc9f",
    "path": "recipes/models/bosonai/higgs-audio-v3-tts-4b.yaml",
    "edit_url": "https://github.com/vllm-project/vllm-omni/edit/main/recipes/models/bosonai/higgs-audio-v3-tts-4b.yaml",
    "overlays": []
  }
}