{ "hf_id": "deepseek-ai/DeepSeek-OCR", "meta": { "title": "DeepSeek-OCR", "slug": "deepseek-ocr", "provider": "DeepSeek", "description": "Frontier OCR model exploring optical context compression for LLMs, optimized for document parsing and markdown generation.", "date_added": "2025-10-22", "date_updated": "2026-04-17", "tasks": [ "multimodal" ], "performance_headline": "Optical context compression for efficient OCR and document understanding", "related_recipes": [ "deepseek-ai/DeepSeek-OCR-2" ], "hardware": { "mi300x": "verified", "mi325x": "verified", "mi355x": "verified", "arc_pro_b60": "verified", "arc_pro_b70": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve deepseek-ai/DeepSeek-OCR \\\n --trust-remote-code \\\n --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor \\\n --mm-processor-cache-gb 0 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "deepseek-ai/DeepSeek-OCR", "--trust-remote-code", "--logits-processors", "vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor", "--mm-processor-cache-gb", "0", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest deepseek-ai/DeepSeek-OCR \\\n --trust-remote-code \\\n --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor \\\n --mm-processor-cache-gb 0 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "deepseek-ai/DeepSeek-OCR", "--trust-remote-code", "--logits-processors", "vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor", "--mm-processor-cache-gb", "0", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy โ€” works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e ยท 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/deepseek-ai/DeepSeek-OCR/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/deepseek-ai/DeepSeek-OCR/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/deepseek-ai/DeepSeek-OCR/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/deepseek-ai/DeepSeek-OCR/hw/h200.json", "h100": "/deepseek-ai/DeepSeek-OCR/hw/h100.json", "b200": "/deepseek-ai/DeepSeek-OCR/hw/b200.json", "gb200": "/deepseek-ai/DeepSeek-OCR/hw/gb200.json", "b300": "/deepseek-ai/DeepSeek-OCR/hw/b300.json", "gb300": "/deepseek-ai/DeepSeek-OCR/hw/gb300.json", "mi300x": "/deepseek-ai/DeepSeek-OCR/hw/mi300x.json", "mi325x": "/deepseek-ai/DeepSeek-OCR/hw/mi325x.json", "mi350x": "/deepseek-ai/DeepSeek-OCR/hw/mi350x.json", "mi355x": "/deepseek-ai/DeepSeek-OCR/hw/mi355x.json", "arc_pro_b60": "/deepseek-ai/DeepSeek-OCR/hw/arc_pro_b60.json", "arc_pro_b70": "/deepseek-ai/DeepSeek-OCR/hw/arc_pro_b70.json" } }, "model": { "model_id": "deepseek-ai/DeepSeek-OCR", "min_vllm_version": "0.12.0", "architecture": "dense", "parameter_count": "3B", "active_parameters": "3B", "context_length": 8192, "base_args": [ "--trust-remote-code", "--logits-processors", "vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor", "--mm-processor-cache-gb", "0" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "text_only": { "description": "Skip loading the vision encoder for text-only workloads โ€” frees VRAM for KV cache. Mutually exclusive with encoder_parallel.", "args": [ "--language-model-only" ] }, "encoder_parallel": { "description": "Run the vision encoder in data-parallel mode โ€” avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.", "args": [ "--mm-encoder-tp-mode", "data" ] } }, "opt_in_features": [ "text_only", "encoder_parallel" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 7, "description": "Full precision BF16 (~3.3B params)" } }, "hardware_overrides": { "amd": { "extra_args": [], "extra_env": { "VLLM_ROCM_USE_AITER": "1", "SAFETENSORS_FAST_GPU": "1" } } }, "guide": "## Overview\n\nDeepSeek-OCR is a frontier OCR model exploring optical context compression for LLMs.\nIt is optimized for document parsing, free-form OCR, and markdown generation from images,\nand ships with a custom n-gram logits processor for optimal quality.\n\n## Prerequisites\n\n- **Hardware**: Single GPU with >=8 GB VRAM (CUDA, MI300X / MI325X / MI355X, or Intel XPU B60 / B70)\n- **vLLM**: Current stable release\n- **Python**: 3.10+ (CUDA) or 3.12 (ROCm)\n\n### CUDA\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\n```\n\n### ROCm (MI300X, MI325X, MI355X)\nRequires Python 3.12, ROCm 7.2.1, and glibc >= 2.35. Use the Docker flow from the\n[vLLM installation guide](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/#pre-built-images)\nif your host environment does not meet those requirements.\n```bash\nuv venv --python 3.12\nsource .venv/bin/activate\nuv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/\n```\n\n### Intel XPU (B60 / B70)\n```bash\ndocker pull vllm/vllm-openai-xpu:latest\n```\n\n## Client Usage\n\n### Offline OCR (Python)\n\n```python\nfrom vllm import LLM, SamplingParams\nfrom vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor\nfrom PIL import Image\n\nllm = LLM(\n model=\"deepseek-ai/DeepSeek-OCR\",\n enable_prefix_caching=False,\n mm_processor_cache_gb=0,\n logits_processors=[NGramPerReqLogitsProcessor],\n)\n\nimage_1 = Image.open(\"path/to/your/image_1.png\").convert(\"RGB\")\nimage_2 = Image.open(\"path/to/your/image_2.png\").convert(\"RGB\")\nprompt = \"\\nFree OCR.\"\n\nmodel_input = [\n {\"prompt\": prompt, \"multi_modal_data\": {\"image\": image_1}},\n {\"prompt\": prompt, \"multi_modal_data\": {\"image\": image_2}},\n]\n\nsampling_param = SamplingParams(\n temperature=0.0,\n max_tokens=8192,\n extra_args=dict(\n ngram_size=30,\n window_size=90,\n whitelist_token_ids={128821, 128822}, # , \n ),\n skip_special_tokens=False,\n)\n\nfor output in llm.generate(model_input, sampling_param):\n print(output.outputs[0].text)\n```\n\n### Online OCR serving\n\n**CUDA**\n```bash\nvllm serve deepseek-ai/DeepSeek-OCR \\\n --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor \\\n --mm-processor-cache-gb 0\n```\n\n**ROCm (MI300X / MI325X / MI355X)**\n```bash\nexport SAFETENSORS_FAST_GPU=1\nexport VLLM_ROCM_USE_AITER=1\n\nvllm serve deepseek-ai/DeepSeek-OCR \\\n --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor \\\n --mm-processor-cache-gb 0\n```\n\n**Intel XPU (B60, B70)**\n\n```bash\ndocker run --device /dev/dri \\\n -v /dev/dri/by-path:/dev/dri/by-path --shm-size=16g \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai-xpu:latest \\\n deepseek-ai/DeepSeek-OCR \\\n --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor \\\n --mm-processor-cache-gb 0\n```\n\nQuery the running server with the OpenAI client (same for CUDA / ROCm / XPU):\n\n```python\nimport time\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\", timeout=3600)\n\nmessages = [\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png\"}},\n {\"type\": \"text\", \"text\": \"Free OCR.\"},\n ],\n }\n]\n\nstart = time.time()\nresponse = client.chat.completions.create(\n model=\"deepseek-ai/DeepSeek-OCR\",\n messages=messages,\n max_tokens=2048,\n temperature=0.0,\n extra_body={\n \"skip_special_tokens\": False,\n \"vllm_xargs\": {\n \"ngram_size\": 30,\n \"window_size\": 90,\n \"whitelist_token_ids\": [128821, 128822],\n },\n },\n)\nprint(f\"Response costs: {time.time() - start:.2f}s\")\nprint(f\"Generated text: {response.choices[0].message.content}\")\n```\n\n## Troubleshooting / Configuration Tips\n\n- **Use the custom logits processor** along with the model for optimal OCR and markdown\n generation performance.\n- Unlike multi-turn chat, OCR tasks do not typically benefit from prefix caching or image\n reuse, so disable these features to avoid unnecessary hashing and caching overhead.\n- DeepSeek-OCR works better with plain prompts than instruction formats. See the\n [official example prompts](https://github.com/deepseek-ai/DeepSeek-OCR/blob/2ac6d64a00656693b79c4f759a5e62c1b78bbeb1/DeepSeek-OCR-master/DeepSeek-OCR-vllm/config.py#L27-L37).\n- Depending on your hardware, adjust `max_num_batched_tokens` for better throughput.\n\n## References\n\n- [DeepSeek-OCR on Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-OCR)\n- [vLLM multimodal inputs guide](https://docs.vllm.ai/en/latest/features/multimodal_inputs.html#offline-inference)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/deepseek-ai/DeepSeek-OCR.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/deepseek-ai/DeepSeek-OCR.yaml", "overlays": [] } }