# pfnet/plamo-3-nict-31b-base — vLLM serving recipe

> Largest PLaMo 3 NICT Japanese/English base model with interleaved sliding-window and full attention.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/pfnet/plamo-3-nict-31b-base
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/pfnet/plamo-3-nict-31b-base.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp`, `multi_node_tp_pp` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.12.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve pfnet/plamo-3-nict-31b-base \
  --trust-remote-code \
  --max-model-len 4096 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest pfnet/plamo-3-nict-31b-base \
  --trust-remote-code \
  --max-model-len 4096 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[PLaMo 3 NICT 31B Base](https://huggingface.co/pfnet/plamo-3-nict-31b-base)
is the largest PLaMo 3 NICT base model from Preferred Networks and NICT.
It is pretrained on Japanese and English data with a hybrid attention stack
combining sliding-window and full-attention layers.

This is a base checkpoint. Use completion-style prompts as-is, or fine-tune
it before deploying instruction-following or chat-style workflows.

### Prerequisites

- **Hardware:** H100/H200 80 GB for 4K context; use TP=2 for extra KV-cache headroom
- **vLLM:** >= 0.12.0
- **License:** gated Hugging Face access under the PLaMo community license

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend=auto
hf auth login
```

### Launching the Server

```bash
vllm serve pfnet/plamo-3-nict-31b-base \
  --trust-remote-code \
  --max-model-len 4096
```

With tensor parallelism for more KV-cache headroom:

```bash
vllm serve pfnet/plamo-3-nict-31b-base \
  --trust-remote-code \
  --max-model-len 4096 \
  --tensor-parallel-size 2
```

### Client Usage

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.completions.create(
    model="pfnet/plamo-3-nict-31b-base",
    prompt="The future of artificial intelligence technology is ",
    max_tokens=128,
    temperature=0.7,
)
print(resp.choices[0].text)
```

### Troubleshooting

- **Out of memory on 80 GB GPUs:** add `--tensor-parallel-size 2` or lower
  `--max-model-len`.
- **Gated repo errors:** accept the PLaMo community license on Hugging Face and
  run `hf auth login`.
- **Repetitive or low-quality completions:** adjust sampling for free-form
  generation, or fine-tune the model for chat or assistant workloads.

### References

- [Model card](https://huggingface.co/pfnet/plamo-3-nict-31b-base)
- [vLLM PLaMo3 support](https://docs.vllm.ai/en/stable/models/supported_models.html)