{ "hf_id": "internlm/Intern-S1", "meta": { "title": "Intern-S1", "slug": "intern-s1", "provider": "InternLM", "description": "Intern-S1 vision-language model from Shanghai AI Lab with BF16/FP8 variants and thinking/non-thinking modes", "date_added": "2025-08-19", "date_updated": "2026-04-17", "tasks": [ "multimodal" ], "related_recipes": [ "OpenGVLab/InternVL3_5-8B" ], "hardware": { "mi300x": "verified", "mi325x": "verified", "mi355x": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve internlm/Intern-S1 \\\n --trust-remote-code \\\n --tensor-parallel-size 8 \\\n --enable-auto-tool-choice \\\n --tool-call-parser internlm \\\n --reasoning-parser deepseek_r1 \\\n --mm-encoder-tp-mode data", "argv": [ "vllm", "serve", "internlm/Intern-S1", "--trust-remote-code", "--tensor-parallel-size", "8", "--enable-auto-tool-choice", "--tool-call-parser", "internlm", "--reasoning-parser", "deepseek_r1", "--mm-encoder-tp-mode", "data" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest internlm/Intern-S1 \\\n --trust-remote-code \\\n --tensor-parallel-size 8 \\\n --enable-auto-tool-choice \\\n --tool-call-parser internlm \\\n --reasoning-parser deepseek_r1 \\\n --mm-encoder-tp-mode data", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "internlm/Intern-S1", "--trust-remote-code", "--tensor-parallel-size", "8", "--enable-auto-tool-choice", "--tool-call-parser", "internlm", "--reasoning-parser", "deepseek_r1", "--mm-encoder-tp-mode", "data" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/internlm/Intern-S1/strategies/multi_node_tp.json", "multi_node_tp_pp": "/internlm/Intern-S1/strategies/multi_node_tp_pp.json", "multi_node_tep": "/internlm/Intern-S1/strategies/multi_node_tep.json", "multi_node_dep": "/internlm/Intern-S1/strategies/multi_node_dep.json", "kv_store_centralized_mooncake": "/internlm/Intern-S1/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/internlm/Intern-S1/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/internlm/Intern-S1/hw/h200.json", "h100": "/internlm/Intern-S1/hw/h100.json", "b200": "/internlm/Intern-S1/hw/b200.json", "gb200": "/internlm/Intern-S1/hw/gb200.json", "b300": "/internlm/Intern-S1/hw/b300.json", "gb300": "/internlm/Intern-S1/hw/gb300.json", "mi300x": "/internlm/Intern-S1/hw/mi300x.json", "mi325x": "/internlm/Intern-S1/hw/mi325x.json", "mi350x": "/internlm/Intern-S1/hw/mi350x.json", "mi355x": "/internlm/Intern-S1/hw/mi355x.json" } }, "model": { "model_id": "internlm/Intern-S1", "min_vllm_version": "0.10.2", "architecture": "moe", "parameter_count": "241B", "active_parameters": "28B", "context_length": 65536, "base_args": [ "--trust-remote-code" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "tool_calling": { "description": "InternLM tool-call parser with automatic tool choice", "args": [ "--enable-auto-tool-choice", "--tool-call-parser", "internlm" ] }, "reasoning": { "description": "DeepSeek-R1 reasoning parser extracts ...", "args": [ "--reasoning-parser", "deepseek_r1" ] }, "text_only": { "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.", "args": [ "--language-model-only" ] }, "encoder_parallel": { "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.", "args": [ "--mm-encoder-tp-mode", "data" ] } }, "opt_in_features": [ "text_only" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 578, "tp": 8, "description": "BF16 on 8x H800 (80GB each)" }, "fp8": { "model_id": "internlm/Intern-S1-FP8", "precision": "fp8", "vram_minimum_gb": 289, "tp": 4, "description": "FP8 on 4x H800 (80GB each)", "json": "/internlm/Intern-S1-FP8.json" } }, "hardware_overrides": { "amd": { "extra_args": [], "extra_env": { "VLLM_ROCM_USE_AITER": "1", "VLLM_ROCM_USE_AITER_MOE": "0" } } }, "guide": "## Overview\n\n[Intern-S1](https://github.com/InternLM/Intern-S1) is a vision-language model developed\nby Shanghai AI Laboratory. It supports thinking and non-thinking modes via chat-template\nkwargs and ships in BF16 and FP8 variants.\n\n## Prerequisites\n\n- Hardware: 8xH800 (80GB) for BF16, 4xH800 for FP8, or 4-8x MI300X/MI325X/MI355X\n- vLLM >= 0.10.2\n\n### Install vLLM (CUDA)\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\n```\n\n### Install vLLM (AMD ROCm)\n\n```bash\nuv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.14.1/rocm700\n```\n\n## Launch commands\n\nBF16 on 8xH800:\n\n```bash\nvllm serve internlm/Intern-S1 \\\n --trust-remote-code \\\n --tensor-parallel-size 8 \\\n --enable-auto-tool-choice \\\n --reasoning-parser deepseek_r1 \\\n --tool-call-parser internlm\n```\n\nFP8 on 4xH800:\n\n```bash\nvllm serve internlm/Intern-S1-FP8 \\\n --trust-remote-code \\\n --tensor-parallel-size 4 \\\n --enable-auto-tool-choice \\\n --reasoning-parser deepseek_r1 \\\n --tool-call-parser internlm\n```\n\nFP8 on 8xMI300X/MI325X:\n\n```bash\nexport VLLM_ROCM_USE_AITER=1\nexport VLLM_ROCM_USE_AITER_MOE=0\nvllm serve internlm/Intern-S1-FP8 \\\n --trust-remote-code --tensor-parallel-size 8 \\\n --enable-auto-tool-choice --reasoning-parser deepseek_r1 --tool-call-parser internlm\n```\n\nFP8 on 8xMI355X: set only `VLLM_ROCM_USE_AITER=1` (no need to disable AITER MoE).\n\n## Switching Between Thinking and Non-Thinking Modes\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(api_key=\"YOUR_API_KEY\", base_url=\"http://0.0.0.0:8000/v1\")\nmodel_name = client.models.list().data[0].id\n\nresponse = client.chat.completions.create(\n model=model_name,\n messages=[{\"role\": \"user\", \"content\": \"9.11 and 9.8, which is greater?\"}],\n temperature=0.8, top_p=0.8,\n extra_body={\"chat_template_kwargs\": {\"enable_thinking\": False}},\n)\nprint(response)\n```\n\n## Troubleshooting\n\n- `ValueError: No available memory for the cache blocks.` — add `--gpu-memory-utilization 0.95`.\n\n## References\n\n- [Intern-S1 on Hugging Face](https://huggingface.co/internlm/Intern-S1)\n- [Intern-S1-FP8](https://huggingface.co/internlm/Intern-S1-FP8)\n- [Intern-S1 GitHub](https://github.com/InternLM/Intern-S1)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/internlm/Intern-S1.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/internlm/Intern-S1.yaml", "overlays": [] } }