{
  "hf_id": "internlm/Intern-S2-Preview",
  "meta": {
    "title": "Intern-S2-Preview",
    "slug": "intern-s2-preview",
    "provider": "InternLM",
    "description": "Scientific multimodal MoE (36B total / 3B active) continued pre-trained from Qwen3.5 — hybrid linear/full attention, 262K context, MTP-accelerated reasoning. BF16 and FP8 checkpoints.",
    "date_added": "2026-05-15",
    "date_updated": "2026-05-15",
    "tasks": [
      "multimodal",
      "text"
    ],
    "performance_headline": "35B-A3B scientific multimodal foundation model — single-node BF16 with MTP",
    "related_recipes": [
      "internlm/Intern-S1",
      "Qwen/Qwen3.5-35B-A3B"
    ],
    "hardware": {
      "h200": "verified",
      "gb200": "verified"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:latest",
    "command": "vllm serve internlm/Intern-S2-Preview \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --reasoning-parser qwen3 \\\n  --mm-encoder-tp-mode data",
    "argv": [
      "vllm",
      "serve",
      "internlm/Intern-S2-Preview",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "qwen3_coder",
      "--reasoning-parser",
      "qwen3",
      "--mm-encoder-tp-mode",
      "data"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:latest internlm/Intern-S2-Preview \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --reasoning-parser qwen3 \\\n  --mm-encoder-tp-mode data",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:latest",
      "internlm/Intern-S2-Preview",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "qwen3_coder",
      "--reasoning-parser",
      "qwen3",
      "--mm-encoder-tp-mode",
      "data"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "single_node_tep": "/internlm/Intern-S2-Preview/strategies/single_node_tep.json",
      "single_node_dep": "/internlm/Intern-S2-Preview/strategies/single_node_dep.json",
      "multi_node_tp": "/internlm/Intern-S2-Preview/strategies/multi_node_tp.json",
      "multi_node_dep": "/internlm/Intern-S2-Preview/strategies/multi_node_dep.json",
      "multi_node_tep": "/internlm/Intern-S2-Preview/strategies/multi_node_tep.json",
      "kv_store_centralized_mooncake": "/internlm/Intern-S2-Preview/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/internlm/Intern-S2-Preview/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/internlm/Intern-S2-Preview/hw/h200.json",
      "h100": "/internlm/Intern-S2-Preview/hw/h100.json",
      "b200": "/internlm/Intern-S2-Preview/hw/b200.json",
      "gb200": "/internlm/Intern-S2-Preview/hw/gb200.json",
      "b300": "/internlm/Intern-S2-Preview/hw/b300.json",
      "gb300": "/internlm/Intern-S2-Preview/hw/gb300.json",
      "mi300x": "/internlm/Intern-S2-Preview/hw/mi300x.json",
      "mi325x": "/internlm/Intern-S2-Preview/hw/mi325x.json",
      "mi350x": "/internlm/Intern-S2-Preview/hw/mi350x.json",
      "mi355x": "/internlm/Intern-S2-Preview/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "internlm/Intern-S2-Preview",
    "min_vllm_version": "0.22.0",
    "architecture": "moe",
    "parameter_count": "36B",
    "active_parameters": "3B",
    "context_length": 262144,
    "base_args": [
      "--trust-remote-code"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "Enable automatic tool choice with the Qwen3 Coder parser (per official deployment guide)",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "qwen3_coder"
      ]
    },
    "reasoning": {
      "description": "Extract <think>...</think> chain-of-thought via the Qwen3 reasoning parser",
      "args": [
        "--reasoning-parser",
        "qwen3"
      ]
    },
    "spec_decoding": {
      "description": "Shared-weight MTP speculative decoding — 4 draft tokens (recommended in the deployment guide)",
      "args": [
        "--speculative-config",
        "{\"method\":\"mtp\",\"num_speculative_tokens\":4}"
      ]
    },
    "text_only": {
      "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.",
      "args": [
        "--language-model-only"
      ]
    },
    "encoder_parallel": {
      "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.",
      "args": [
        "--mm-encoder-tp-mode",
        "data"
      ]
    }
  },
  "opt_in_features": [
    "spec_decoding",
    "text_only"
  ],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 87,
      "description": "BF16 on 1x H200 or 2x H100/H800 (deployment guide uses TP=2)"
    },
    "fp8": {
      "model_id": "internlm/Intern-S2-Preview-FP8",
      "precision": "fp8",
      "vram_minimum_gb": 44,
      "description": "Official DeepSeek-style block FP8 (128x128, ue8m0 scales) — fits on a single H100/H200",
      "json": "/internlm/Intern-S2-Preview-FP8.json"
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\n[Intern-S2-Preview](https://huggingface.co/internlm/Intern-S2-Preview) is a\nscientific multimodal foundation model from Shanghai AI Laboratory, continued\npre-trained from Qwen3.5. It packs 36B total / 3B active parameters across\n256 experts with hybrid linear/full attention, supports a 262K context, and\nships with built-in shared-weight MTP for fast reasoning.\n\nBeyond chat and reasoning, it adds vision and time-series modalities and\nimproves agent capabilities for scientific workflows.\n\n## Prerequisites\n\n- **vLLM version:** nightly build with `InternS2PreviewForConditionalGeneration`\n  support ([PR #42705](https://github.com/vllm-project/vllm/pull/42705)). The\n  architecture is not yet in any stable release.\n- **Hardware (BF16):** 1x H200 (141 GB) or 2x H100/H800 (80 GB)\n- **Hardware (FP8):** single H100/H200\n- **Trust remote code:** required (custom modeling files ship in the repo)\n\n### Install vLLM (nightly)\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly\n```\n\n## Launch commands\n\n### Recommended — with MTP speculative decoding\n\n```bash\nvllm serve internlm/Intern-S2-Preview \\\n  --trust-remote-code \\\n  --tensor-parallel-size 2 \\\n  --reasoning-parser qwen3 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":4}'\n```\n\n### Basic serving (no MTP)\n\n```bash\nvllm serve internlm/Intern-S2-Preview \\\n  --trust-remote-code \\\n  --tensor-parallel-size 2 \\\n  --reasoning-parser qwen3 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder\n```\n\n### Long-context (YaRN, up to 512K)\n\nThe base config sets `max_position_embeddings = 262144`. For longer contexts,\noverride the RoPE config to enable YaRN:\n\n```bash\nvllm serve internlm/Intern-S2-Preview \\\n  --trust-remote-code \\\n  --tensor-parallel-size 2 \\\n  --max-model-len 512000 \\\n  --reasoning-parser qwen3 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --hf-overrides '{\"text_config\": {\"rope_parameters\": {\"mrope_interleaved\": true, \"mrope_section\": [11, 11, 10], \"rope_type\": \"yarn\", \"rope_theta\": 10000000, \"partial_rotary_factor\": 0.25, \"factor\": 4.0, \"original_max_position_embeddings\": 262144}}}'\n```\n\n## Client Usage\n\nRecommended sampling parameters from the model card:\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\nresp = client.chat.completions.create(\n    model=\"internlm/Intern-S2-Preview\",\n    messages=[{\"role\": \"user\", \"content\": \"Design a synthesis route for paracetamol.\"}],\n    temperature=0.8,\n    top_p=0.95,\n    max_tokens=32768,\n    extra_body={\n        \"top_k\": 50,\n        \"min_p\": 0.0,\n        \"spaces_between_special_tokens\": False,\n    },\n)\nprint(resp.choices[0].message.content)\n```\n\n### Toggle thinking mode\n\nThinking is enabled by default. Disable it per request:\n\n```python\nresp = client.chat.completions.create(\n    model=\"internlm/Intern-S2-Preview\",\n    messages=[{\"role\": \"user\", \"content\": \"What is AGI?\"}],\n    temperature=0.8,\n    top_p=0.95,\n    extra_body={\"chat_template_kwargs\": {\"enable_thinking\": False}},\n)\n```\n\n> The model card notes: do **not** disable thinking mode for agentic tasks.\n\n## Troubleshooting\n\n- **Unknown architecture `InternS2PreviewForConditionalGeneration`:** the\n  handler landed via vLLM PR #42705 — use a nightly wheel or build from main\n  once the PR merges.\n- **OOM at full 262K context:** drop `--max-model-len` to 65536 or 131072,\n  or lower `--gpu-memory-utilization` headroom.\n\n## References\n\n- [Model card](https://huggingface.co/internlm/Intern-S2-Preview)\n- [Deployment guide](https://huggingface.co/internlm/Intern-S2-Preview/blob/main/deployment_guide.md)\n- [Intern-S1 GitHub](https://github.com/InternLM/Intern-S1)\n- [vLLM support PR #42705](https://github.com/vllm-project/vllm/pull/42705)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "389ffebe555705c85337025d5df108d0384d365e",
    "path": "recipes/models/internlm/Intern-S2-Preview.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/internlm/Intern-S2-Preview.yaml",
    "overlays": []
  }
}