# poolside/Laguna-XS.2 — vLLM serving recipe

> Poolside's 33B total / 3B activated MoE coding model with mixed sliding-window + global attention, native interleaved reasoning, and 256K context — designed for agentic coding.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/poolside/Laguna-XS.2
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/poolside/Laguna-XS.2.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `fp8`, `nvfp4`, `int4`, `redhat_fp8`, `redhat_nvfp4`
- `strategy`: `single_node_tp`, `single_node_tep`, `single_node_dep` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `spec_decoding` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.21.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve poolside/Laguna-XS.2 \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest poolside/Laguna-XS.2 \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Laguna XS.2](https://huggingface.co/poolside/Laguna-XS.2) is Poolside's 33B-total / 3B-activated Mixture-of-Experts model purpose-built for agentic coding and long-horizon work. It combines mixed sliding-window + global attention (3:1 across 40 layers) with sigmoid per-head gating and FP8 KV cache, so it stays compact enough to run locally while supporting a 256K-token context.

#### Key features
- **Mixed SWA + global attention**: 30 sliding-window layers (window=512) interleaved with 10 global-attention layers, each with per-layer rotary scaling.
- **Native FP8 KV cache**: KV cache is quantized to FP8 to reduce memory per token.
- **Interleaved reasoning**: thinking blocks emitted between tool calls; toggled per-request via `enable_thinking`.
- **Tool calling**: Poolside-specific XML-style tool-call protocol, parsed via `poolside_v1`.
- **256 experts + 1 shared expert** with top-8 routing.

### Prerequisites

Laguna XS.2 support is on the open vLLM PR ([vllm-project/vllm#41129](https://github.com/vllm-project/vllm/pull/41129)) — install from a nightly wheel or the pinned Docker image below until the PR lands in a stable release.

#### Docker (recommended)
```bash
docker pull vllm/vllm-openai:laguna
```

#### pip (nightly)
```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --pre \
  --extra-index-url https://wheels.vllm.ai/nightly/cu130 \
  --extra-index-url https://download.pytorch.org/whl/cu130 \
  --index-strategy unsafe-best-match
```

### Quantized variants

Pre-quantized checkpoints are published alongside the BF16 weights — select one with the **Variant** control above:

| Variant | Checkpoint | Weights | ~VRAM | Notes |
|---------|-----------|---------|-------|-------|
| BF16 | [Laguna-XS.2](https://huggingface.co/poolside/Laguna-XS.2) | BF16 | 80 GB | Default |
| FP8 | [Laguna-XS.2-FP8](https://huggingface.co/poolside/Laguna-XS.2-FP8) | Block-FP8 | 40 GB | Set `VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER=0` |
| NVFP4 | [Laguna-XS.2-NVFP4](https://huggingface.co/poolside/Laguna-XS.2-NVFP4) | NVFP4 | 26 GB | Blackwell only |
| INT4 | [Laguna-XS.2-INT4](https://huggingface.co/poolside/Laguna-XS.2-INT4) | INT4 (W4A16) | 29 GB | Hopper or Blackwell |

Quantization is detected automatically from each checkpoint's `quantization_config`, so the launch command is identical — just swap the model ID (select the variant above). All three quantized checkpoints ship an **FP8-quantized KV cache**.

> **vLLM ≥ 0.22.0 required for the quantized variants.** The FP8 KV cache needs the per-layer attention-head fix in [vllm#42650](https://github.com/vllm-project/vllm/pull/42650); earlier versions produce scrambled output on non-Hopper GPUs. On older vLLM you can disable the FP8 KV cache with `--kv-cache-dtype-skip-layers $(seq 0 39)`.

### Deployment Configurations

#### Single GPU (H100/H200/B200, BF16)
```bash
vllm serve poolside/Laguna-XS.2 \
  --trust-remote-code \
  --max-model-len 262144 \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1
```

#### Docker
```bash
docker run -itd --name laguna-xs2 \
  --ipc=host --network host --shm-size 16G --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:laguna \
    --model poolside/Laguna-XS.2 \
    --trust-remote-code \
    --max-model-len 262144 \
    --enable-auto-tool-choice \
    --tool-call-parser poolside_v1 \
    --reasoning-parser poolside_v1 \
    --host 0.0.0.0 --port 8000
```

#### AMD MI300X / MI325X / MI355X

```bash
vllm serve poolside/Laguna-XS.2 \
  --trust-remote-code \
  --max-model-len 262144 \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1
```

#### Docker (AMD MI300X / MI325X / MI355X)

```bash
docker run --rm -it \
  --cap-add=SYS_PTRACE \
  --ipc=host \
  --privileged=true \
  --shm-size=128GB \
  --network=host \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add video \
  vllm/vllm-openai-rocm:latest \
    --model poolside/Laguna-XS.2 \
    --trust-remote-code \
    --max-model-len 262144 \
    --enable-auto-tool-choice \
    --tool-call-parser poolside_v1 \
    --reasoning-parser poolside_v1 
```

### Controlling reasoning

Reasoning is **off by default** in the chat template. Enable it per-request:

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

resp = client.chat.completions.create(
    model="poolside/Laguna-XS.2",
    messages=[{"role": "user", "content": "Write a Python retry wrapper with exponential backoff."}],
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
    temperature=0.7,
    top_p=1.0,
    extra_query={"top_k": 20},
)
print(resp.choices[0].message.reasoning)
print(resp.choices[0].message.content)
```

Or default-on with `--default-chat-template-kwargs '{"enable_thinking": true}'`.

### Speculative decoding (DFlash)

Enable the **Spec Decoding** toggle above to attach Poolside's [DFlash draft model](https://huggingface.co/poolside/Laguna-XS.2-speculator.dflash) — a 5-layer Llama-style speculator that proposes up to 7 tokens per step. Reported per-position acceptance with reasoning enabled is ~70% at position 1 across coding, math, QA, and writing workloads.

Requires:
- vLLM built from [PR #41880](https://github.com/vllm-project/vllm/pull/41880) (extends the base Laguna PR with DFlash support).
- `VLLM_USE_DEEP_GEMM=0` in the launch environment — DeepGEMM is currently incompatible with the DFlash draft path.

Example:

```bash
VLLM_USE_DEEP_GEMM=0 vllm serve poolside/Laguna-XS.2 \
  --trust-remote-code \
  --max-model-len 16384 \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1 \
  --speculative-config '{"model":"poolside/Laguna-XS.2-speculator.dflash","num_speculative_tokens":7,"method":"dflash"}'
```

### References

- [Model card](https://huggingface.co/poolside/Laguna-XS.2)
- [Release blog post](https://poolside.ai/blog/laguna-a-deeper-dive)
- [vLLM support PR #41129](https://github.com/vllm-project/vllm/pull/41129)
- [vLLM DFlash spec-decoding PR #41880](https://github.com/vllm-project/vllm/pull/41880)
- [DFlash draft model](https://huggingface.co/poolside/Laguna-XS.2-speculator.dflash)