# zai-org/GLM-GA — vLLM serving recipe

> GLM-GA dense vision-language model (~10B) — image and video understanding with 128K context and dedicated Glmga video processor (fps=2, up to 640 frames)

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/zai-org/GLM-GA
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/zai-org/GLM-GA.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `text_only` (default: none)

## Install

Requires vLLM >= 0.21.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

Extra dependencies:

- Latest transformers from main branch for GlmgaImageProcessor / GlmgaVideoProcessor support

```bash
uv pip install git+https://github.com/huggingface/transformers.git
```

## Serve

```bash
export VLLM_VIDEO_LOADER_BACKEND=glm4_6v
vllm serve zai-org/GLM-GA \
  --trust-remote-code \
  --allowed-local-media-path / \
  --mm-processor-cache-type shm \
  --reasoning-parser glm45 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_VIDEO_LOADER_BACKEND=glm4_6v \
  vllm/vllm-openai:latest zai-org/GLM-GA \
  --trust-remote-code \
  --allowed-local-media-path / \
  --mm-processor-cache-type shm \
  --reasoning-parser glm45 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

GLM-GA is a dense vision-language model (~10B parameters) based on the
GLM-4.6V-Flash architecture. It shares the same `Glm4vForConditionalGeneration`
model class as GLM-4.6V but uses dedicated `GlmgaImageProcessor` and
`GlmgaVideoProcessor` sub-processors. The key difference is in video processing:
GLM-GA samples at a fixed 2 fps and supports up to 640 frames, enabling
long-video understanding.

### Prerequisites

- **vLLM version:** >= 0.21.0
- **Transformers:** latest (required for Glmga processor support)
- **Hardware:** 1x H100/H200 or equivalent (BF16)

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
uv pip install git+https://github.com/huggingface/transformers.git
```

### Launching the Server

#### Single GPU

```bash
export VLLM_VIDEO_LOADER_BACKEND=glm4_6v
vllm serve zai-org/GLM-GA \
     --tool-call-parser glm45 \
     --reasoning-parser glm45 \
     --enable-auto-tool-choice \
     --allowed-local-media-path / \
     --mm-processor-cache-type shm
```

#### Tuning Tips

- `--max-model-len=65536` is a common default; you can push to 131072.
- `--max-num-batched-tokens=32768` for prompt-heavy workloads.
- `--mm-processor-cache-type shm` for efficient vision processing.

### Client Usage

#### Image Understanding

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="zai-org/GLM-GA",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}},
            {"type": "text", "text": "Describe the image."}
        ]
    }],
    max_tokens=512,
)
print(resp.choices[0].message.content)
```

#### Video Understanding

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="zai-org/GLM-GA",
    messages=[{
        "role": "user",
        "content": [
            {"type": "video_url", "video_url": {"url": "https://example.com/video.mp4"}},
            {"type": "text", "text": "Summarize what happens in this video."}
        ]
    }],
    max_tokens=1024,
)
print(resp.choices[0].message.content)
```

### Video Processing Details

GLM-GA uses a dedicated `GlmgaVideoProcessor` that differs from GLM-4.6V:
- Fixed 2 fps sampling (GLM-4.6V uses dynamic fps based on duration)
- Up to 640 frames per video
- Frame upsampling uses `math.floor` alignment matching the HuggingFace reference

### Troubleshooting

- **Long-context memory:** At 128K context, tune `--max-num-batched-tokens` and
  `--gpu-memory-utilization` to prevent OOM.
- **Video loading errors:** Ensure OpenCV or PyAV is installed for video decoding.

### References

- [Model card](https://huggingface.co/zai-org/GLM-GA)
- [GLM-4.6V-Flash](https://huggingface.co/zai-org/GLM-4.6V-Flash) (base architecture)
- [vLLM multimodal inputs guide](https://docs.vllm.ai/en/latest/features/multimodal_inputs.html)