# fishaudio/drama-3 — vLLM serving recipe

> Fish Audio's multi-speaker text-to-speech model (35B MoE, 3B active) served via vLLM-Omni, with voice cloning, speaker tags and inline event tags through the OpenAI /v1/audio/speech API.

Configuration: hardware `h100` (8× H100) · variant `default` (FP8) · task `t2a`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/fishaudio/drama-3
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/fishaudio/drama-3.json?hardware=h100

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `task`: `t2a`

## Install

Requires vLLM >= 0.31.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Extra dependencies:

- vLLM-Omni provides the TTS serving backend and the `--omni` serve path. Drama 3 support is in vllm-project/vllm-omni#8900.

```bash
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
vllm serve fishaudio/drama-3 \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3) is a
text-to-speech model built for multi-character scenes and long-form content:
a 35B-total / 3B-active MoE text backbone plus a codebook decoder that
produces 10 codebooks per frame, decoded to **44.1 kHz mono** audio. It
supports zero-shot voice cloning, speaker tags (`<|speaker:0|>` …
`<|speaker:15|>`) and inline event tags such as `[sighs]` or `[laughs]`.

vLLM-Omni serves it as two stages on one GPU: the AR stage (backbone +
codebook decoder) streams frames to the codec stage, which returns audio
through the OpenAI-compatible `/v1/audio/speech` API.

### Prerequisites

- **Hardware**: a single CUDA GPU with at least 48 GB. Reference run:
  1x H100 80 GB (stage-0 weights 36.9 GiB).
- **vLLM-Omni** with Drama 3 support, on vLLM >= 0.31.

### Installation

```bash
uv venv && source .venv/bin/activate
uv pip install vllm==0.31.0 --torch-backend=auto
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

### Launch the server

The deploy config (`vllm_omni/deploy/fish_omni.yaml`) is picked up
automatically from `model_type=fish_omni`:

```bash
vllm serve fishaudio/drama-3 --omni --port 8000
```

The first request after start compiles kernels and takes about 10 s;
send a few warmup requests before measuring latency.

### Client usage

#### Voice cloning (reference audio + transcript)

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fishaudio/drama-3",
    "input": "The quarterly report is due on Thursday, so please finish it by noon.",
    "voice": "default",
    "ref_audio": "https://huggingface.co/datasets/zhaochenyang20/seed-tts-eval-mini/resolve/main/en/prompt-wavs/common_voice_en_10119832.wav",
    "ref_text": "We asked over twenty different people, and they all said it was his.",
    "response_format": "wav"
  }' --output cloned.wav
```

#### Streaming

Add `"stream": true, "response_format": "pcm"` to receive Server-Sent
Events whose `speech.audio.delta` events carry base64 16-bit PCM at 44.1 kHz.

#### Event and emotion tags

Non-verbal events and delivery are controlled only through the input text.
Write a short natural-language descriptor in square brackets where the event
should happen:

```text
[sighs] Fine. If we miss it, it's on you. [laughs] Well, maybe that's fair.
```

- There is no fixed tag list. Tags are ordinary text (no special tokens), so
  any short descriptor is accepted; wordings close to the ones Fish Audio
  documents work best: `[sighs]`, `[laughs]`, `[laughing]`, `[whisper]`,
  `[angry]`, `[excited]`, `[pause]`.
- Speaker tags `<|speaker:0|>` … `<|speaker:15|>` are the only special control
  tokens: `"<|speaker:0|> Hi. <|speaker:1|> Hello."`. Text without a leading
  speaker tag is spoken by speaker 0.
- The reference clip also shapes delivery implicitly (an expressive or
  pause-heavy reference carries over), but it is not a targeted control.
- A tag or filler word at the very start of a line (`[sighs] …`, `Yeah. …`)
  more often renders the first word or event and then only silence until
  `max_new_tokens`; mid-sentence tags are more reliable.

### Performance

1x H100 80 GB, both stages on one GPU, streaming `/v1/audio/speech`, one
voice-clone reference, temperature 1.0 / top_p 0.9 / top_k 30, warm server:

| Concurrency | First audio (p50) | RTF |
|---:|---:|---:|
| 1 | 126 ms | 0.28 |
| 4 | 246 ms | 0.40 |
| 8 | 360 ms | 0.51 |

Voice cloning, 5 public Common Voice reference voices x 4 English
sentences: Whisper-large-v3 WER 0.000.

### Known limitations

- Output is **44.1 kHz mono**; voice cloning requires **both** `ref_audio`
  and `ref_text`.
- Some takes finish speaking and then keep emitting silence until
  `max_new_tokens`, most often on prompts that start with fillers or event
  tags; cap `max_new_tokens` to the expected length of the text.
- Editing as continuation (`previous_takes`) and long-text chunking are not
  exposed through the speech API yet.

### References

- [Model card](https://huggingface.co/fishaudio/drama-3)
- vLLM-Omni support: vllm-project/vllm-omni [#8900](https://github.com/vllm-project/vllm-omni/pull/8900)