# tencent/HunyuanOCR — vLLM serving recipe

> Tencent Hunyuan end-to-end OCR expert VLM (~1B) for online OCR serving with an OpenAI-compatible API

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/tencent/HunyuanOCR
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/tencent/HunyuanOCR.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `text_only`, `encoder_parallel` (default: none)

## Install

Requires vLLM >= 0.12.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve tencent/HunyuanOCR \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest tencent/HunyuanOCR \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[HunyuanOCR](https://huggingface.co/tencent/HunyuanOCR) is a leading end-to-end OCR
expert VLM powered by Hunyuan's native multimodal architecture. This recipe covers
online serving with the OpenAI-compatible API.

### Prerequisites

- **vLLM version:** latest stable
- **Hardware:** single GPU (1B model)

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

### Launching the Server

```bash
vllm serve tencent/HunyuanOCR \
    --no-enable-prefix-caching \
    --mm-processor-cache-gb 0
```

#### Configuration Tips

- Use greedy sampling (`temperature=0.0`) or low temperature for optimal OCR accuracy.
- OCR tasks generally do not benefit from prefix caching or image reuse; disabling
  them (as above) removes hashing/caching overhead.
- Adjust `--max-num-batched-tokens` for throughput based on your hardware.

### Client Usage

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=3600)

messages = [
    {"role": "system", "content": ""},
    {
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/chat-ui/tools-dark.png"}},
            {
                "type": "text",
                "text": (
                    "Extract all information from the main body of the document image "
                    "and represent it in markdown format, ignoring headers and footers. "
                    "Tables should be expressed in HTML format, formulas in the document "
                    "should be represented using LaTeX format, and the parsing should be "
                    "organized according to the reading order."
                )
            }
        ]
    }
]

response = client.chat.completions.create(
    model="tencent/HunyuanOCR",
    messages=messages,
    temperature=0.0,
    extra_body={"top_k": 1, "repetition_penalty": 1.0},
)
print(response.choices[0].message.content)
```

### Troubleshooting

- **Accuracy:** Use `temperature=0.0` and `top_k=1` for deterministic OCR output.
- **Application-oriented prompts:** See the [official model card](https://huggingface.co/tencent/HunyuanOCR#%F0%9F%92%AC-application-oriented-prompts)
  for prompts tuned to various document parsing tasks.

### References

- [Model card](https://huggingface.co/tencent/HunyuanOCR)