# internlm/Intern-S2-Preview — vLLM serving recipe

> Scientific multimodal MoE (36B total / 3B active) continued pre-trained from Qwen3.5 — hybrid linear/full attention, 262K context, MTP-accelerated reasoning. BF16 and FP8 checkpoints.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`, `encoder_parallel`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/internlm/Intern-S2-Preview
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/internlm/Intern-S2-Preview.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `fp8`
- `strategy`: `single_node_tp`, `single_node_tep`, `single_node_dep`, `multi_node_tp`, `multi_node_dep`, `multi_node_tep` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `spec_decoding`, `text_only`, `encoder_parallel` (default: `tool_calling`, `reasoning`, `encoder_parallel`)

## Install

Requires vLLM >= 0.22.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve internlm/Intern-S2-Preview \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest internlm/Intern-S2-Preview \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Intern-S2-Preview](https://huggingface.co/internlm/Intern-S2-Preview) is a
scientific multimodal foundation model from Shanghai AI Laboratory, continued
pre-trained from Qwen3.5. It packs 36B total / 3B active parameters across
256 experts with hybrid linear/full attention, supports a 262K context, and
ships with built-in shared-weight MTP for fast reasoning.

Beyond chat and reasoning, it adds vision and time-series modalities and
improves agent capabilities for scientific workflows.

### Prerequisites

- **vLLM version:** nightly build with `InternS2PreviewForConditionalGeneration`
  support ([PR #42705](https://github.com/vllm-project/vllm/pull/42705)). The
  architecture is not yet in any stable release.
- **Hardware (BF16):** 1x H200 (141 GB) or 2x H100/H800 (80 GB)
- **Hardware (FP8):** single H100/H200
- **Trust remote code:** required (custom modeling files ship in the repo)

#### Install vLLM (nightly)

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
```

### Launch commands

#### Recommended — with MTP speculative decoding

```bash
vllm serve internlm/Intern-S2-Preview \
  --trust-remote-code \
  --tensor-parallel-size 2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"mtp","num_speculative_tokens":4}'
```

#### Basic serving (no MTP)

```bash
vllm serve internlm/Intern-S2-Preview \
  --trust-remote-code \
  --tensor-parallel-size 2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder
```

#### Long-context (YaRN, up to 512K)

The base config sets `max_position_embeddings = 262144`. For longer contexts,
override the RoPE config to enable YaRN:

```bash
vllm serve internlm/Intern-S2-Preview \
  --trust-remote-code \
  --tensor-parallel-size 2 \
  --max-model-len 512000 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}'
```

### Client Usage

Recommended sampling parameters from the model card:

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Preview",
    messages=[{"role": "user", "content": "Design a synthesis route for paracetamol."}],
    temperature=0.8,
    top_p=0.95,
    max_tokens=32768,
    extra_body={
        "top_k": 50,
        "min_p": 0.0,
        "spaces_between_special_tokens": False,
    },
)
print(resp.choices[0].message.content)
```

#### Toggle thinking mode

Thinking is enabled by default. Disable it per request:

```python
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Preview",
    messages=[{"role": "user", "content": "What is AGI?"}],
    temperature=0.8,
    top_p=0.95,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
```

> The model card notes: do **not** disable thinking mode for agentic tasks.

### Troubleshooting

- **Unknown architecture `InternS2PreviewForConditionalGeneration`:** the
  handler landed via vLLM PR #42705 — use a nightly wheel or build from main
  once the PR merges.
- **OOM at full 262K context:** drop `--max-model-len` to 65536 or 131072,
  or lower `--gpu-memory-utilization` headroom.

### References

- [Model card](https://huggingface.co/internlm/Intern-S2-Preview)
- [Deployment guide](https://huggingface.co/internlm/Intern-S2-Preview/blob/main/deployment_guide.md)
- [Intern-S1 GitHub](https://github.com/InternLM/Intern-S1)
- [vLLM support PR #42705](https://github.com/vllm-project/vllm/pull/42705)