# OpenMOSS-Team/MOSS-TTS-Realtime — vLLM serving recipe

> OpenMOSS's 1.7B real-time streaming TTS for low-latency voice agents (TTFB ~180 ms) — multi-turn context-aware incremental synthesis — served via vLLM-Omni through the OpenAI /v1/audio/speech API (24 kHz mono).

Configuration: hardware `h200` (8× H200) · variant `default` (BF16) · task `t2a`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/OpenMOSS-Team/MOSS-TTS-Realtime
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/OpenMOSS-Team/MOSS-TTS-Realtime.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `task`: `t2a`

## Install

Requires vLLM >= 0.22.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Extra dependencies:

- vLLM-Omni provides the TTS serving backend and the `--omni` serve path. See the installation guide for pre-built wheels or building from source.

```bash
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
vllm serve OpenMOSS-Team/MOSS-TTS-Realtime \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[MOSS-TTS-Realtime](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Realtime) is
the **1.7B real-time streaming** member of OpenMOSS's MOSS-TTS Family, served
through **vLLM-Omni** with the OpenAI-compatible `/v1/audio/speech` API. It is
a multi-turn, context-aware model for **low-latency voice agents**, using a
local depth transformer (no delay warm-up) for incremental synthesis. TTFB
(time to first byte) is **~180 ms**, and LLM-first-sentence + TTS-TTFB is
~377 ms. Output is **24 kHz mono**.

### Prerequisites

- **Hardware**: a single CUDA GPU. Reference run: 1x A10G 24 GB — ~6 GB for the
  1.7B talker + ~8 GB for the codec decoder.
- **vLLM-Omni** targeting vLLM >= 0.22.

### Installation

```bash
uv venv && source .venv/bin/activate
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

### Launch the server

MOSS-TTS-Realtime uses `model_type=moss_tts_realtime`, so its deploy config
(`vllm_omni/deploy/moss_tts_realtime.yaml`, `codec_chunk_frames: 15` for low
TTFA) is auto-discovered:

```bash
vllm serve OpenMOSS-Team/MOSS-TTS-Realtime --omni --port 8000
```

### Client usage

#### Streaming voice clone (curl)

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "OpenMOSS-Team/MOSS-TTS-Realtime",
    "input": "This is a low-latency streaming TTS test.",
    "voice": "default",
    "ref_audio": "https://raw.githubusercontent.com/OpenMOSS/MOSS-TTS/main/assets/audio/zh_1.wav",
    "response_format": "wav",
    "stream": true
  }' --output output.wav
```

### Known limitations

- Output is **24 kHz mono**.
- Shares the `OpenMOSS-Team/MOSS-Audio-Tokenizer` codec (~7 GB, auto-downloaded;
  override with `MOSS_TTS_CODEC_PATH`).

### References

- [Model card](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Realtime)
- [MOSS-TTS Family](https://huggingface.co/OpenMOSS-Team/MOSS-TTS) · [GitHub](https://github.com/OpenMOSS/MOSS-TTS)