# JetBrains/Mellum2-12B-A2.5B-Thinking — vLLM serving recipe

> JetBrains' reasoning-augmented code MoE (12B total / 2.5B active) that emits explicit <think> chains for debugging, planning, and agentic coding

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `reasoning`, `tool_calling`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/JetBrains/Mellum2-12B-A2.5B-Thinking
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/JetBrains/Mellum2-12B-A2.5B-Thinking.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `fp8`, `nvfp4`
- `strategy`: `single_node_tp`, `single_node_tep`, `single_node_dep` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `reasoning`, `tool_calling`, `spec_decoding` (default: `reasoning`, `tool_calling`)

## Install

Requires vLLM >= 0.23.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --tensor-parallel-size 1 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest JetBrains/Mellum2-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --tensor-parallel-size 1 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Mellum2-12B-A2.5B-Thinking](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Thinking)
is JetBrains' reasoning-augmented code assistant. It uses a Mixture-of-Experts architecture
with 64 experts (8 activated per token) — 12B total parameters, 2.5B active — combining
sliding-window and full-attention layers for a 131,072-token context. The model emits its
chain-of-thought inside `<think>...</think>` blocks before the final answer, making it
suited to complex debugging, multi-step planning, and agentic workflows. For direct,
low-latency answers without reasoning traces, use the
[Instruct](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct) variant instead.

### Prerequisites

- Hardware: a single H200, H100, or A100 (~29 GB at bf16) is plenty
- vLLM **nightly** — `MellumForCausalLM` support landed after v0.22.0 and is not yet in a
  stable release. Install the nightly wheels until the next tagged release ships.

#### Install vLLM (nightly)

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --extra-index-url https://wheels.vllm.ai/nightly
```

### Launch command

```bash
# With reasoning (recommended for the Thinking checkpoint)
vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --reasoning-parser qwen3

# Add tool calling
vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes
```

### Client usage

JetBrains recommends sampling at `temperature=0.6`, `top_p=0.95`, `top_k=20` for the
Thinking checkpoint.

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

resp = client.chat.completions.create(
    model="JetBrains/Mellum2-12B-A2.5B-Thinking",
    messages=[{"role": "user", "content": "Is 1024 a power of 2? Explain your reasoning."}],
    max_tokens=81920,
    temperature=0.6,
    top_p=0.95,
    extra_body={"top_k": 20},
)
print(resp.choices[0].message.content)
```

### References

- [Model card](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Thinking)
- [vLLM support PR #43992](https://github.com/vllm-project/vllm/pull/43992)
- [Mellum2 Technical Report](https://arxiv.org/abs/2605.31268)