{
  "hf_id": "fishaudio/drama-3",
  "meta": {
    "title": "Fish Audio Drama 3",
    "slug": "fish-audio-drama-3",
    "provider": "Fish Audio",
    "description": "Fish Audio's multi-speaker text-to-speech model (35B MoE, 3B active) served via vLLM-Omni, with voice cloning, speaker tags and inline event tags through the OpenAI /v1/audio/speech API.",
    "date_added": "2026-10-13",
    "date_updated": "2026-10-13",
    "tasks": [
      "omni"
    ],
    "performance_headline": "1x H100: first audio 126 ms, RTF 0.28 at concurrency 1",
    "related_recipes": [
      "fishaudio/s2-pro",
      "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice"
    ],
    "default_hardware": "h100",
    "hardware": {
      "h100": "verified"
    }
  },
  "model": {
    "model_id": "fishaudio/drama-3",
    "min_vllm_version": "0.31.0",
    "architecture": "moe",
    "parameter_count": "35B",
    "active_parameters": "3B",
    "context_length": 0,
    "base_args": [
      "--omni"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      },
      "extras": [
        {
          "note": "vLLM-Omni provides the TTS serving backend and the `--omni` serve path. Drama 3 support is in vllm-project/vllm-omni#8900.",
          "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git"
        }
      ]
    }
  },
  "omni": {
    "tasks": [
      {
        "id": "t2a",
        "label": "Text → Speech",
        "endpoint": "/v1/audio/speech",
        "description": "Drama 3 voice-cloned TTS via /v1/audio/speech (44.1 kHz mono)",
        "curl": "curl -X POST http://localhost:8000/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"fishaudio/drama-3\",\n    \"input\": \"The quarterly report is due on Thursday, so please finish it by noon.\",\n    \"voice\": \"default\",\n    \"ref_audio\": \"https://huggingface.co/datasets/zhaochenyang20/seed-tts-eval-mini/resolve/main/en/prompt-wavs/common_voice_en_10119832.wav\",\n    \"ref_text\": \"We asked over twenty different people, and they all said it was his.\",\n    \"response_format\": \"wav\"\n  }' --output output.wav\n"
      }
    ]
  },
  "features": {},
  "opt_in_features": [],
  "variants": {
    "default": {
      "precision": "fp8",
      "vram_minimum_gb": 48,
      "description": "Block-FP8 routed experts, other weights BF16 — single GPU (stage-0 weights 36.9 GiB; reference run on 1x H100 80 GB)"
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\n[Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3) is a\ntext-to-speech model built for multi-character scenes and long-form content:\na 35B-total / 3B-active MoE text backbone plus a codebook decoder that\nproduces 10 codebooks per frame, decoded to **44.1 kHz mono** audio. It\nsupports zero-shot voice cloning, speaker tags (`<|speaker:0|>` …\n`<|speaker:15|>`) and inline event tags such as `[sighs]` or `[laughs]`.\n\nvLLM-Omni serves it as two stages on one GPU: the AR stage (backbone +\ncodebook decoder) streams frames to the codec stage, which returns audio\nthrough the OpenAI-compatible `/v1/audio/speech` API.\n\n## Prerequisites\n\n- **Hardware**: a single CUDA GPU with at least 48 GB. Reference run:\n  1x H100 80 GB (stage-0 weights 36.9 GiB).\n- **vLLM-Omni** with Drama 3 support, on vLLM >= 0.31.\n\n## Installation\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install vllm==0.31.0 --torch-backend=auto\nuv pip install git+https://github.com/vllm-project/vllm-omni.git\n```\n\n## Launch the server\n\nThe deploy config (`vllm_omni/deploy/fish_omni.yaml`) is picked up\nautomatically from `model_type=fish_omni`:\n\n```bash\nvllm serve fishaudio/drama-3 --omni --port 8000\n```\n\nThe first request after start compiles kernels and takes about 10 s;\nsend a few warmup requests before measuring latency.\n\n## Client usage\n\n### Voice cloning (reference audio + transcript)\n\n```bash\ncurl -X POST http://localhost:8000/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"fishaudio/drama-3\",\n    \"input\": \"The quarterly report is due on Thursday, so please finish it by noon.\",\n    \"voice\": \"default\",\n    \"ref_audio\": \"https://huggingface.co/datasets/zhaochenyang20/seed-tts-eval-mini/resolve/main/en/prompt-wavs/common_voice_en_10119832.wav\",\n    \"ref_text\": \"We asked over twenty different people, and they all said it was his.\",\n    \"response_format\": \"wav\"\n  }' --output cloned.wav\n```\n\n### Streaming\n\nAdd `\"stream\": true, \"response_format\": \"pcm\"` to receive Server-Sent\nEvents whose `speech.audio.delta` events carry base64 16-bit PCM at 44.1 kHz.\n\n### Event and emotion tags\n\nNon-verbal events and delivery are controlled only through the input text.\nWrite a short natural-language descriptor in square brackets where the event\nshould happen:\n\n```text\n[sighs] Fine. If we miss it, it's on you. [laughs] Well, maybe that's fair.\n```\n\n- There is no fixed tag list. Tags are ordinary text (no special tokens), so\n  any short descriptor is accepted; wordings close to the ones Fish Audio\n  documents work best: `[sighs]`, `[laughs]`, `[laughing]`, `[whisper]`,\n  `[angry]`, `[excited]`, `[pause]`.\n- Speaker tags `<|speaker:0|>` … `<|speaker:15|>` are the only special control\n  tokens: `\"<|speaker:0|> Hi. <|speaker:1|> Hello.\"`. Text without a leading\n  speaker tag is spoken by speaker 0.\n- The reference clip also shapes delivery implicitly (an expressive or\n  pause-heavy reference carries over), but it is not a targeted control.\n- A tag or filler word at the very start of a line (`[sighs] …`, `Yeah. …`)\n  more often renders the first word or event and then only silence until\n  `max_new_tokens`; mid-sentence tags are more reliable.\n\n## Performance\n\n1x H100 80 GB, both stages on one GPU, streaming `/v1/audio/speech`, one\nvoice-clone reference, temperature 1.0 / top_p 0.9 / top_k 30, warm server:\n\n| Concurrency | First audio (p50) | RTF |\n|---:|---:|---:|\n| 1 | 126 ms | 0.28 |\n| 4 | 246 ms | 0.40 |\n| 8 | 360 ms | 0.51 |\n\nVoice cloning, 5 public Common Voice reference voices x 4 English\nsentences: Whisper-large-v3 WER 0.000.\n\n## Known limitations\n\n- Output is **44.1 kHz mono**; voice cloning requires **both** `ref_audio`\n  and `ref_text`.\n- Some takes finish speaking and then keep emitting silence until\n  `max_new_tokens`, most often on prompts that start with fillers or event\n  tags; cap `max_new_tokens` to the expected length of the text.\n- Editing as continuation (`previous_takes`) and long-text chunking are not\n  exposed through the speech API yet.\n\n## References\n\n- [Model card](https://huggingface.co/fishaudio/drama-3)\n- vLLM-Omni support: vllm-project/vllm-omni [#8900](https://github.com/vllm-project/vllm-omni/pull/8900)\n",
  "source": {
    "source": "vllm-omni",
    "repo": "vllm-project/vllm-omni",
    "ref": "main",
    "sha": "cda4a1ef83b295ba332146a7709e6277725edc9f",
    "path": "recipes/models/fishaudio/drama-3.yaml",
    "edit_url": "https://github.com/vllm-project/vllm-omni/edit/main/recipes/models/fishaudio/drama-3.yaml",
    "overlays": []
  }
}