# mistralai/Voxtral-4B-TTS-2603 — vLLM serving recipe

> Mistral's 4B text-to-speech model served via vLLM-Omni with built-in voice presets, exposed through the OpenAI /v1/audio/speech API (24 kHz mono).

Configuration: hardware `h200` (8× H200) · variant `default` (BF16) · task `t2a`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/mistralai/Voxtral-4B-TTS-2603
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/mistralai/Voxtral-4B-TTS-2603.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `task`: `t2a`

## Install

Requires vLLM >= 0.20.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Extra dependencies:

- vLLM-Omni provides the TTS serving backend and the `--omni` serve path. See the installation guide for pre-built wheels or building from source.

```bash
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

- Voxtral's tokenizer/audio path requires mistral_common >= 1.10.0

```bash
uv pip install -U "mistral_common>=1.10.0"
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
vllm serve mistralai/Voxtral-4B-TTS-2603 \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Voxtral-4B-TTS-2603](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603) is
Mistral AI's 4B text-to-speech model, served through **vLLM-Omni** with the
OpenAI-compatible `/v1/audio/speech` API. It generates **24 kHz mono** audio
from text using model-provided voice presets.

### Prerequisites

- **Hardware**: a single CUDA GPU with **≥ 24 GB** VRAM. Reference run: 1×
  RTX 4090 24 GB — Stage 0 (`audio_generation`) ~18.95 GiB, Stage 1
  (`audio_tokenizer`) ~1.55 GiB, server-startup peak ~20.5 GiB (~85%). Both
  stages share GPU 0 via the bundled deploy config.
- **vLLM-Omni** targeting vLLM >= 0.20, with `mistral_common >= 1.10.0`.

### Installation

```bash
uv venv && source .venv/bin/activate
uv pip install git+https://github.com/vllm-project/vllm-omni.git
uv pip install -U "mistral_common>=1.10.0"
```

### Launch the server

The deploy config (`vllm_omni/deploy/voxtral_tts.yaml`) is auto-discovered:

```bash
vllm serve mistralai/Voxtral-4B-TTS-2603 --omni --port 8000
```

### Client usage

#### Voice preset (curl)

```bash
curl -X POST http://127.0.0.1:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistralai/Voxtral-4B-TTS-2603",
    "input": "Hello, this is Voxtral TTS running with vLLM-Omni.",
    "voice": "casual_female",
    "language": "English",
    "response_format": "wav"
  }' --output voxtral.wav
```

#### Streaming PCM

```bash
curl -X POST http://127.0.0.1:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistralai/Voxtral-4B-TTS-2603",
    "input": "Hello, this is Voxtral TTS streaming PCM.",
    "voice": "casual_female",
    "language": "English",
    "stream": true,
    "response_format": "pcm"
  }' --output voxtral_stream.pcm

# Convert raw 24 kHz mono PCM to WAV
ffmpeg -f s16le -ar 24000 -ac 1 -i voxtral_stream.pcm voxtral_stream.wav -y
```

#### Examples

- [Offline end-to-end](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/text_to_speech/voxtral_tts/end2end.py)
- [Online Gradio demo](https://github.com/vllm-project/vllm-omni/blob/main/examples/online_serving/text_to_speech/voxtral_tts/gradio_demo.py)

### Known limitations

- Output is **mono 24 kHz**.
- **Voice cloning is gated upstream.** With the public
  `mistralai/Voxtral-4B-TTS-2603` checkpoint, supplying a `ref_audio` (or
  running the voice-clone path) fails with
  `RuntimeError: encode_waveforms requires encoder weights which are not
  available in the open-source checkpoint.` — the public weights omit the
  encoder needed to turn reference audio into conditioning features. Use the
  built-in voice presets instead.

### References

- [Model card](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603)
- [vLLM-Omni text-to-speech examples](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_speech/voxtral_tts)