# PaddlePaddle/PaddleOCR-VL-1.6 — vLLM serving recipe

> PaddleOCR-VL-1.6 (0.9B) — region-aware data optimization + progressive post-training; new SOTA 96.33% on OmniDocBench v1.6, drop-in replacement for 1.5

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/PaddlePaddle/PaddleOCR-VL-1.6
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/PaddlePaddle/PaddleOCR-VL-1.6.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `text_only`, `encoder_parallel` (default: none)

## Install

Requires vLLM >= 0.11.1.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

Extra dependencies:

- PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)

```bash
uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/
```

- PaddleOCR document-parsing helpers — 1.6 requires >= 3.6.0 (pipeline_version v1.6)

```bash
uv pip install -U "paddleocr[doc-parser]>=3.6.0"
```

- Safetensors loader used by the doc-parser path

```bash
uv pip install safetensors
```

## Serve

```bash
vllm serve PaddlePaddle/PaddleOCR-VL-1.6 \
  --trust-remote-code \
  --max-num-batched-tokens 16384 \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest PaddlePaddle/PaddleOCR-VL-1.6 \
  --trust-remote-code \
  --max-num-batched-tokens 16384 \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[PaddleOCR-VL-1.6](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6) upgrades
PaddleOCR-VL-1.5 with a region-aware data optimization framework (targeted enhancement
of weak regions found in the previous model) and a progressive post-training recipe
based on curated data selection and reinforcement learning:

- **96.33% on OmniDocBench v1.6** (new SOTA), plus new records on OmniDocBench v1.5
  and Real5-OmniDocBench (scanning, warping, screen-photography, illumination, skew)
- Major gains in table, Chinese ancient document, and Chinese rare character recognition
- Improved seal/stamp recognition, text spotting, and chart recognition

The architecture is **fully compatible with PaddleOCR-VL-1.5** (same 0.9B NaViT-style
dynamic-resolution vision encoder + ERNIE-4.5-0.3B LM), so migration is plug-and-play:
the vLLM launch command and feature flags are unchanged.

### Prerequisites

- Hardware: 1x GPU (small VRAM footprint)
- vLLM >= 0.11.1

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm
```

### Launch command

```bash
vllm serve PaddlePaddle/PaddleOCR-VL-1.6 \
  --trust-remote-code \
  --max-num-batched-tokens 16384 \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0
```

Tip: OCR workloads don't benefit much from prefix caching or image reuse, so disable
those to avoid hashing/caching overhead.

### Client Usage

Task-specific prompts (same task set as 1.5, including `spotting` and `seal`):

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=3600)

TASKS = {
    "ocr": "OCR:",
    "table": "Table Recognition:",
    "formula": "Formula Recognition:",
    "chart": "Chart Recognition:",
    "spotting": "Spotting:",
    "seal": "Seal Recognition:",
}

response = client.chat.completions.create(
    model="PaddlePaddle/PaddleOCR-VL-1.6",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://.../receipt.png"}},
            {"type": "text", "text": TASKS["spotting"]},
        ],
    }],
    temperature=0.0,
)
print(response.choices[0].message.content)
```

### Offline Inference with PP-DocLayoutV3

Use separate venvs for `vllm` and `paddlepaddle` to avoid conflicts. The 1.6 pipeline
requires `paddleocr[doc-parser] >= 3.6.0` and `pipeline_version="v1.6"`, which
automatically uses [PP-DocLayoutV3](https://huggingface.co/PaddlePaddle/PP-DocLayoutV3)
for layout detection (PP-DocLayoutV2 belongs to the v1 pipeline). Pass
`vl_rec_api_model_name="PaddlePaddle/PaddleOCR-VL-1.6"` so the client matches the
model name the vLLM server exposes — otherwise it defaults to
`PaddleOCR-VL-1.6-0.9B` and fails with "The model PaddleOCR-VL-1.6-0.9B does not exist."

```bash
uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/
uv pip install -U "paddleocr[doc-parser]>=3.6.0"
uv pip install safetensors
```

```python
from paddleocr import PaddleOCRVL

pipeline = PaddleOCRVL(
    pipeline_version="v1.6",
    vl_rec_backend="vllm-server",
    vl_rec_server_url="http://localhost:8000/v1",
    vl_rec_api_model_name="PaddlePaddle/PaddleOCR-VL-1.6",
)

output = pipeline.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/paddleocr_vl_demo.png")
for i, res in enumerate(output):
    res.save_to_json(save_path=f"output_{i}.json")
    res.save_to_markdown(save_path=f"output_{i}.md")
```

### References

- [PaddleOCR-VL-1.6 on Hugging Face](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6)
- [Technical report (arXiv)](https://arxiv.org/pdf/2606.03264)
- [PaddleOCR GitHub](https://github.com/PaddlePaddle/PaddleOCR)
- [vLLM support PR](https://github.com/vllm-project/vllm/pull/27758)