{ "hf_id": "Google/translategemma-27b-it", "meta": { "title": "TranslateGemma 27B IT", "slug": "translategemma-27b-it", "provider": "Google", "description": "Lightweight open translation model from Google (based on Gemma 3) supporting 55 languages. Served via the vLLM-optimized Infomaniak-AI checkpoint.", "date_added": "2026-03-06", "date_updated": "2026-04-17", "tasks": [ "text" ], "performance_headline": "Deployable on laptops/desktops and cloud GPUs; vLLM-optimized checkpoint removes custom JSON inputs", "related_recipes": [] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve google/translategemma-27b-it \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "google/translategemma-27b-it", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest google/translategemma-27b-it \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "google/translategemma-27b-it", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy โ€” works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e ยท 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/Google/translategemma-27b-it/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/Google/translategemma-27b-it/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/Google/translategemma-27b-it/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/Google/translategemma-27b-it/hw/h200.json", "h100": "/Google/translategemma-27b-it/hw/h100.json", "b200": "/Google/translategemma-27b-it/hw/b200.json", "gb200": "/Google/translategemma-27b-it/hw/gb200.json", "b300": "/Google/translategemma-27b-it/hw/b300.json", "gb300": "/Google/translategemma-27b-it/hw/gb300.json", "mi300x": "/Google/translategemma-27b-it/hw/mi300x.json", "mi325x": "/Google/translategemma-27b-it/hw/mi325x.json", "mi350x": "/Google/translategemma-27b-it/hw/mi350x.json", "mi355x": "/Google/translategemma-27b-it/hw/mi355x.json" } }, "model": { "model_id": "google/translategemma-27b-it", "min_vllm_version": "0.14.1", "architecture": "dense", "parameter_count": "27B", "active_parameters": "27B", "context_length": 131072, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": {}, "opt_in_features": [], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 65, "description": "Original Google checkpoint in BF16 (has vLLM compatibility issues โ€” prefer vllm-optimized variant)" }, "vllm_optimized": { "model_id": "Infomaniak-AI/vllm-translategemma-27b-it", "precision": "bf16", "vram_minimum_gb": 65, "description": "Infomaniak-AI vLLM-optimized checkpoint โ€” recommended. Fixes RoPE config, EOS token, and replaces custom JSON inputs with string delimiters.", "json": "/Infomaniak-AI/vllm-translategemma-27b-it.json" }, "small_4b": { "model_id": "google/translategemma-4b-it", "precision": "bf16", "vram_minimum_gb": 10, "description": "4B variant for lower-resource deployments (prefer Infomaniak-AI/vllm-translategemma-4b-it)", "json": "/google/translategemma-4b-it.json" } }, "hardware_overrides": {}, "guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models from Google, based on the Gemma 3 family. TranslateGemma models handle translation across 55 languages and are small enough to deploy on laptops, desktops, and modest cloud GPU environments.\n\n### Original Models\n- [google/translategemma-27b-it](https://huggingface.co/google/translategemma-27b-it)\n- [google/translategemma-4b-it](https://huggingface.co/google/translategemma-4b-it)\n\n### Optimized vLLM Models\n- [Infomaniak-AI/vllm-translategemma-27b-it](https://huggingface.co/Infomaniak-AI/vllm-translategemma-27b-it)\n- [Infomaniak-AI/vllm-translategemma-4b-it](https://huggingface.co/Infomaniak-AI/vllm-translategemma-4b-it)\n\n### Why use the vLLM-optimized models?\n\nThe original Google models have compatibility issues with standard inference engines like vLLM. The optimized versions from Infomaniak-AI ([detailed changes](https://huggingface.co/Infomaniak-AI/vllm-translategemma-27b-it#changes-from-original-model)):\n\n- **vLLM Compatibility**: Originals require custom JSON parameters (`source_lang_code`/`target_lang_code`). The optimized version uses string delimiters.\n- **RoPE Simplification**: Originals use a complex RoPE configuration for sliding attention. Optimized uses a standard linear RoPE format (`factor: 8.0`).\n- **EOS Token Fix**: Corrects the EOS token from `` to ``.\n\n## Prerequisites\n\n### Docker\n```bash\ndocker pull vllm/vllm-openai:latest\n```\n\n## Deployment Configurations\n\nVerified for both 4B and 27B:\n```bash\ndocker run -itd --name google-translategemma-27b-it \\\n --ipc=host \\\n --network host \\\n --shm-size 16G \\\n --gpus all \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest \\\n Infomaniak-AI/vllm-translategemma-27b-it \\\n --served-model-name translategemma-27b-it \\\n --gpu-memory-utilization 0.8 \\\n --host 0.0.0.0 \\\n --port 8000\n```\n\n## Client Usage\n\nTips:\n- **Prompt Delimiters**: Encode language metadata directly in the content string:\n `<<>>{src_lang}<<>>{tgt_lang}<<>>{text}`\n- **Language Codes**: ISO 639-1 Alpha-2 (e.g. `en`, `zh`) and regional variants (e.g. `en_US`, `zh_CN`).\n- **Context Limit**: ~2K tokens.\n\n### cURL\n```bash\ncurl -X POST http://localhost:8000/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"translategemma-27b-it\",\n \"messages\": [{\n \"role\": \"user\",\n \"content\": \"<<>>en<<>>zh<<>>We distribute two models for language identification, which can recognize 176 languages.\"\n }]\n }'\n```\n\n## References\n\n- [TranslateGemma collection](https://huggingface.co/collections/google/translategemma)\n- [Infomaniak-AI optimized 27B](https://huggingface.co/Infomaniak-AI/vllm-translategemma-27b-it)\n- [Infomaniak-AI optimized 4B](https://huggingface.co/Infomaniak-AI/vllm-translategemma-4b-it)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/Google/translategemma-27b-it.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/Google/translategemma-27b-it.yaml", "overlays": [] } }