{ "hf_id": "tencent/HunyuanOCR", "meta": { "title": "HunyuanOCR", "slug": "hunyuan-ocr", "provider": "Hunyuan (Tencent)", "description": "Tencent Hunyuan end-to-end OCR expert VLM (~1B) for online OCR serving with an OpenAI-compatible API", "date_added": "2025-11-25", "date_updated": "2026-04-17", "tasks": [ "multimodal" ], "performance_headline": "Compact 1B end-to-end OCR VLM from the Hunyuan native multimodal family", "related_recipes": [] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve tencent/HunyuanOCR \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "tencent/HunyuanOCR", "--no-enable-prefix-caching", "--mm-processor-cache-gb", "0", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest tencent/HunyuanOCR \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "tencent/HunyuanOCR", "--no-enable-prefix-caching", "--mm-processor-cache-gb", "0", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/tencent/HunyuanOCR/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/tencent/HunyuanOCR/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/tencent/HunyuanOCR/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/tencent/HunyuanOCR/hw/h200.json", "h100": "/tencent/HunyuanOCR/hw/h100.json", "b200": "/tencent/HunyuanOCR/hw/b200.json", "gb200": "/tencent/HunyuanOCR/hw/gb200.json", "b300": "/tencent/HunyuanOCR/hw/b300.json", "gb300": "/tencent/HunyuanOCR/hw/gb300.json", "mi300x": "/tencent/HunyuanOCR/hw/mi300x.json", "mi325x": "/tencent/HunyuanOCR/hw/mi325x.json", "mi350x": "/tencent/HunyuanOCR/hw/mi350x.json", "mi355x": "/tencent/HunyuanOCR/hw/mi355x.json" } }, "model": { "model_id": "tencent/HunyuanOCR", "min_vllm_version": "0.12.0", "architecture": "dense", "parameter_count": "1B", "active_parameters": "1B", "context_length": 32768, "base_args": [ "--no-enable-prefix-caching", "--mm-processor-cache-gb", "0" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "text_only": { "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.", "args": [ "--language-model-only" ] }, "encoder_parallel": { "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.", "args": [ "--mm-encoder-tp-mode", "data" ] } }, "opt_in_features": [ "text_only", "encoder_parallel" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 2, "description": "Full precision BF16 — single-GPU deployment" } }, "hardware_overrides": {}, "guide": "## Overview\n\n[HunyuanOCR](https://huggingface.co/tencent/HunyuanOCR) is a leading end-to-end OCR\nexpert VLM powered by Hunyuan's native multimodal architecture. This recipe covers\nonline serving with the OpenAI-compatible API.\n\n## Prerequisites\n\n- **vLLM version:** latest stable\n- **Hardware:** single GPU (1B model)\n\n### Install vLLM\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\n```\n\n## Launching the Server\n\n```bash\nvllm serve tencent/HunyuanOCR \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0\n```\n\n### Configuration Tips\n\n- Use greedy sampling (`temperature=0.0`) or low temperature for optimal OCR accuracy.\n- OCR tasks generally do not benefit from prefix caching or image reuse; disabling\n them (as above) removes hashing/caching overhead.\n- Adjust `--max-num-batched-tokens` for throughput based on your hardware.\n\n## Client Usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\", timeout=3600)\n\nmessages = [\n {\"role\": \"system\", \"content\": \"\"},\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/chat-ui/tools-dark.png\"}},\n {\n \"type\": \"text\",\n \"text\": (\n \"Extract all information from the main body of the document image \"\n \"and represent it in markdown format, ignoring headers and footers. \"\n \"Tables should be expressed in HTML format, formulas in the document \"\n \"should be represented using LaTeX format, and the parsing should be \"\n \"organized according to the reading order.\"\n )\n }\n ]\n }\n]\n\nresponse = client.chat.completions.create(\n model=\"tencent/HunyuanOCR\",\n messages=messages,\n temperature=0.0,\n extra_body={\"top_k\": 1, \"repetition_penalty\": 1.0},\n)\nprint(response.choices[0].message.content)\n```\n\n## Troubleshooting\n\n- **Accuracy:** Use `temperature=0.0` and `top_k=1` for deterministic OCR output.\n- **Application-oriented prompts:** See the [official model card](https://huggingface.co/tencent/HunyuanOCR#%F0%9F%92%AC-application-oriented-prompts)\n for prompts tuned to various document parsing tasks.\n\n## References\n\n- [Model card](https://huggingface.co/tencent/HunyuanOCR)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/tencent/HunyuanOCR.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/tencent/HunyuanOCR.yaml", "overlays": [] } }