{ "hf_id": "zai-org/GLM-GA", "meta": { "title": "GLM-GA", "slug": "glm-ga", "provider": "GLM (Z-AI)", "description": "GLM-GA dense vision-language model (~10B) — image and video understanding with 128K context and dedicated Glmga video processor (fps=2, up to 640 frames)", "date_added": "2026-06-03", "date_updated": "2026-05-27", "tasks": [ "multimodal" ], "performance_headline": "Dense VLM based on GLM-4.6V-Flash with dedicated video processor supporting long videos up to 640 frames", "related_recipes": [ "zai-org/GLM-4.6V" ] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": { "VLLM_VIDEO_LOADER_BACKEND": "glm4_6v" }, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve zai-org/GLM-GA \\\n --trust-remote-code \\\n --allowed-local-media-path / \\\n --mm-processor-cache-type shm \\\n --reasoning-parser glm45 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "zai-org/GLM-GA", "--trust-remote-code", "--allowed-local-media-path", "/", "--mm-processor-cache-type", "shm", "--reasoning-parser", "glm45", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n -e VLLM_VIDEO_LOADER_BACKEND=glm4_6v \\\n vllm/vllm-openai:latest zai-org/GLM-GA \\\n --trust-remote-code \\\n --allowed-local-media-path / \\\n --mm-processor-cache-type shm \\\n --reasoning-parser glm45 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "-e", "VLLM_VIDEO_LOADER_BACKEND=glm4_6v", "vllm/vllm-openai:latest", "zai-org/GLM-GA", "--trust-remote-code", "--allowed-local-media-path", "/", "--mm-processor-cache-type", "shm", "--reasoning-parser", "glm45", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/zai-org/GLM-GA/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/zai-org/GLM-GA/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/zai-org/GLM-GA/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/zai-org/GLM-GA/hw/h200.json", "h100": "/zai-org/GLM-GA/hw/h100.json", "b200": "/zai-org/GLM-GA/hw/b200.json", "gb200": "/zai-org/GLM-GA/hw/gb200.json", "b300": "/zai-org/GLM-GA/hw/b300.json", "gb300": "/zai-org/GLM-GA/hw/gb300.json", "mi300x": "/zai-org/GLM-GA/hw/mi300x.json", "mi325x": "/zai-org/GLM-GA/hw/mi325x.json", "mi350x": "/zai-org/GLM-GA/hw/mi350x.json", "mi355x": "/zai-org/GLM-GA/hw/mi355x.json" } }, "model": { "model_id": "zai-org/GLM-GA", "min_vllm_version": "0.21.0", "architecture": "dense", "parameter_count": "10B", "context_length": 131072, "base_args": [ "--trust-remote-code", "--allowed-local-media-path", "/", "--mm-processor-cache-type", "shm", "--reasoning-parser", "glm45" ], "base_env": { "VLLM_VIDEO_LOADER_BACKEND": "glm4_6v" }, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" }, "extras": [ { "command": "uv pip install git+https://github.com/huggingface/transformers.git", "note": "Latest transformers from main branch for GlmgaImageProcessor / GlmgaVideoProcessor support" } ] } }, "features": { "text_only": { "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache.", "args": [ "--language-model-only" ] } }, "opt_in_features": [ "text_only" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 29, "description": "Full precision BF16 — runs on a single H100/H200" } }, "hardware_overrides": {}, "guide": "## Overview\n\nGLM-GA is a dense vision-language model (~10B parameters) based on the\nGLM-4.6V-Flash architecture. It shares the same `Glm4vForConditionalGeneration`\nmodel class as GLM-4.6V but uses dedicated `GlmgaImageProcessor` and\n`GlmgaVideoProcessor` sub-processors. The key difference is in video processing:\nGLM-GA samples at a fixed 2 fps and supports up to 640 frames, enabling\nlong-video understanding.\n\n## Prerequisites\n\n- **vLLM version:** >= 0.21.0\n- **Transformers:** latest (required for Glmga processor support)\n- **Hardware:** 1x H100/H200 or equivalent (BF16)\n\n### Install vLLM\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\nuv pip install git+https://github.com/huggingface/transformers.git\n```\n\n## Launching the Server\n\n### Single GPU\n\n```bash\nexport VLLM_VIDEO_LOADER_BACKEND=glm4_6v\nvllm serve zai-org/GLM-GA \\\n --tool-call-parser glm45 \\\n --reasoning-parser glm45 \\\n --enable-auto-tool-choice \\\n --allowed-local-media-path / \\\n --mm-processor-cache-type shm\n```\n\n### Tuning Tips\n\n- `--max-model-len=65536` is a common default; you can push to 131072.\n- `--max-num-batched-tokens=32768` for prompt-heavy workloads.\n- `--mm-processor-cache-type shm` for efficient vision processing.\n\n## Client Usage\n\n### Image Understanding\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\nresp = client.chat.completions.create(\n model=\"zai-org/GLM-GA\",\n messages=[{\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/image.png\"}},\n {\"type\": \"text\", \"text\": \"Describe the image.\"}\n ]\n }],\n max_tokens=512,\n)\nprint(resp.choices[0].message.content)\n```\n\n### Video Understanding\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\nresp = client.chat.completions.create(\n model=\"zai-org/GLM-GA\",\n messages=[{\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"video_url\", \"video_url\": {\"url\": \"https://example.com/video.mp4\"}},\n {\"type\": \"text\", \"text\": \"Summarize what happens in this video.\"}\n ]\n }],\n max_tokens=1024,\n)\nprint(resp.choices[0].message.content)\n```\n\n## Video Processing Details\n\nGLM-GA uses a dedicated `GlmgaVideoProcessor` that differs from GLM-4.6V:\n- Fixed 2 fps sampling (GLM-4.6V uses dynamic fps based on duration)\n- Up to 640 frames per video\n- Frame upsampling uses `math.floor` alignment matching the HuggingFace reference\n\n## Troubleshooting\n\n- **Long-context memory:** At 128K context, tune `--max-num-batched-tokens` and\n `--gpu-memory-utilization` to prevent OOM.\n- **Video loading errors:** Ensure OpenCV or PyAV is installed for video decoding.\n\n## References\n\n- [Model card](https://huggingface.co/zai-org/GLM-GA)\n- [GLM-4.6V-Flash](https://huggingface.co/zai-org/GLM-4.6V-Flash) (base architecture)\n- [vLLM multimodal inputs guide](https://docs.vllm.ai/en/latest/features/multimodal_inputs.html)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/zai-org/GLM-GA.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/zai-org/GLM-GA.yaml", "overlays": [] } }