{ "hf_id": "zai-org/Glyph", "meta": { "title": "Glyph", "slug": "glyph", "provider": "GLM (Z-AI)", "description": "Visual-text compression framework that renders long text into images and processes them with a reasoning VLM, scaling effective context length", "date_added": "2025-10-28", "date_updated": "2026-04-17", "tasks": [ "multimodal" ], "performance_headline": "Reasoning multimodal model for visual-text compression, single-GPU deployable", "related_recipes": [], "hardware": { "h100": "verified", "mi300x": "verified", "mi325x": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve zai-org/Glyph \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0 \\\n --limit-mm-per-prompt.video 0 \\\n --tensor-parallel-size 1 \\\n --reasoning-parser glm45", "argv": [ "vllm", "serve", "zai-org/Glyph", "--no-enable-prefix-caching", "--mm-processor-cache-gb", "0", "--limit-mm-per-prompt.video", "0", "--tensor-parallel-size", "1", "--reasoning-parser", "glm45" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest zai-org/Glyph \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0 \\\n --limit-mm-per-prompt.video 0 \\\n --tensor-parallel-size 1 \\\n --reasoning-parser glm45", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "zai-org/Glyph", "--no-enable-prefix-caching", "--mm-processor-cache-gb", "0", "--limit-mm-per-prompt.video", "0", "--tensor-parallel-size", "1", "--reasoning-parser", "glm45" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/zai-org/Glyph/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/zai-org/Glyph/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/zai-org/Glyph/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/zai-org/Glyph/hw/h200.json", "h100": "/zai-org/Glyph/hw/h100.json", "b200": "/zai-org/Glyph/hw/b200.json", "gb200": "/zai-org/Glyph/hw/gb200.json", "b300": "/zai-org/Glyph/hw/b300.json", "gb300": "/zai-org/Glyph/hw/gb300.json", "mi300x": "/zai-org/Glyph/hw/mi300x.json", "mi325x": "/zai-org/Glyph/hw/mi325x.json", "mi350x": "/zai-org/Glyph/hw/mi350x.json", "mi355x": "/zai-org/Glyph/hw/mi355x.json" } }, "model": { "model_id": "zai-org/Glyph", "min_vllm_version": "0.11.0", "architecture": "dense", "parameter_count": "10B", "active_parameters": "10B", "context_length": 131072, "base_args": [ "--no-enable-prefix-caching", "--mm-processor-cache-gb", "0", "--limit-mm-per-prompt.video", "0" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "reasoning": { "description": "GLM-4.5 reasoning parser for extracting reasoning traces from Glyph outputs", "args": [ "--reasoning-parser", "glm45" ] }, "text_only": { "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.", "args": [ "--language-model-only" ] }, "encoder_parallel": { "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.", "args": [ "--mm-encoder-tp-mode", "data" ] } }, "opt_in_features": [ "text_only", "encoder_parallel" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 24, "description": "Full precision BF16 — single-GPU deployment on 1xH100" } }, "kv_offload_support": { "offloading_cpu": "verified", "offloading_fs": "verified" }, "hardware_overrides": {}, "guide": "## Overview\n\n[Glyph](https://github.com/thu-coai/Glyph) is a framework from Zhipu AI for scaling\ncontext length via visual-text compression. It renders long textual sequences into\nimages and processes them with a vision-language model. This recipe covers the vLLM\ndeployment of the `zai-org/Glyph` VLM as a component in that framework.\n\nGlyph is a reasoning multimodal model, so `--reasoning-parser glm45` is recommended\nto parse reasoning traces from outputs.\n\n## Prerequisites\n\n- **vLLM version:** latest stable\n- **Hardware:** 1x H100 or 1x MI300X/MI325X\n\n### Install vLLM (NVIDIA)\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\n```\n\n### Install vLLM (AMD ROCm)\n\n```bash\nuv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm\n```\n\nROCm wheel requires Python 3.12, ROCm 7.0, glibc >= 2.35.\n\n## Launching the Server\n\n### Single H100 GPU\n\n```bash\nvllm serve zai-org/Glyph \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0 \\\n --reasoning-parser glm45 \\\n --limit-mm-per-prompt.video 0\n```\n\n### Single MI300X / MI325X\n\n```bash\nVLLM_ROCM_USE_AITER=1 \\\nSAFETENSORS_FAST_GPU=1 \\\nvllm serve zai-org/Glyph \\\n --no-enable-prefix-caching \\\n --mm-processor-cache-gb 0 \\\n --reasoning-parser glm45 \\\n --limit-mm-per-prompt.video 0\n```\n\n### Configuration Tips\n\n- `--no-enable-prefix-caching` and `--mm-processor-cache-gb 0` are recommended for\n OCR-like workloads where image reuse is uncommon; they avoid unnecessary hashing\n and caching overhead.\n- Adjust `--max-num-batched-tokens` for throughput according to your hardware.\n\n## Benchmarking\n\n```bash\nvllm bench serve \\\n --model zai-org/Glyph \\\n --dataset-name random \\\n --random-input-len 8192 \\\n --random-output-len 512 \\\n --request-rate 10000 \\\n --num-prompts 16 \\\n --ignore-eos\n```\n\n## Troubleshooting\n\n- **Reasoning traces:** Use `--reasoning-parser glm45` to extract reasoning content.\n- **Slow first inference:** Disabling prefix caching and multimodal processor caching\n is intentional for Glyph's use case and trades off first-request latency for predictable throughput.\n\n## References\n\n- [Model card](https://huggingface.co/zai-org/Glyph)\n- [Glyph framework on GitHub](https://github.com/thu-coai/Glyph)\n- [vLLM deployment section of Glyph docs](https://github.com/thu-coai/Glyph?tab=readme-ov-file#model-deployment-vllm-acceleration)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/zai-org/Glyph.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/zai-org/Glyph.yaml", "overlays": [] } }