# jinaai/jina-reranker-m0 — vLLM serving recipe

> Multilingual, multimodal reranker for text and visual documents across 29+ languages via Qwen2-VL backbone

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/jinaai/jina-reranker-m0
- This configuration as JSON: https://vllm-recipes-cimvhk5sf-inferact-inc.vercel.app/jinaai/jina-reranker-m0.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.10.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve jinaai/jina-reranker-m0 \
  --gpu-memory-utilization 0.75 \
  --max-num-seqs 32 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest jinaai/jina-reranker-m0 \
  --gpu-memory-utilization 0.75 \
  --max-num-seqs 32 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[jinaai/jina-reranker-m0](https://huggingface.co/jinaai/jina-reranker-m0) is a
multilingual, multimodal reranker that ranks visual documents across 29+ languages.
It accepts text and visual content, including pages with mixed text, figures, tables,
and various layouts.

Deployment target: 2x NVIDIA T4 or 2x NVIDIA L4.

### Prerequisites

- Hardware: 2x T4 or 2x L4 (or any 2x GPU with ~16 GB each)
- vLLM >= 0.10.0

#### Install vLLM (CUDA)

```bash
uv pip install vllm
```

#### Install vLLM (AMD ROCm)

```bash
uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.14.1/rocm700
```

### Launch command

```bash
vllm serve jinaai/jina-reranker-m0 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor_parallel_size 2 \
  --gpu-memory-utilization 0.75 \
  --max-num-seqs 32
```

On AMD:

```bash
export VLLM_ROCM_USE_AITER=1
vllm serve jinaai/jina-reranker-m0 \
  --tensor_parallel_size 2 --gpu-memory-utilization 0.75 --max-num-seqs 32
```

### Rerank API

```bash
curl -X POST http://localhost:8000/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jinaai/jina-reranker-m0",
    "query": "What are the health benefits of green tea?",
    "documents": [
      "Green tea contains antioxidants called catechins...",
      "El precio del café ha aumentado un 20% este año...",
      "Studies show that drinking green tea regularly..."
    ],
    "top_n": 3,
    "return_documents": true
  }'
```

### Score API

Text-to-text:

```bash
curl -X POST http://localhost:8000/v1/score \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jinaai/jina-reranker-m0",
    "text_1": ["What is the capital of Brazil?"],
    "text_2": ["The capital of Brazil is Brasilia."]
  }'
```

Multimodal (text vs. images):

```json
{
  "model": "jinaai/jina-reranker-m0",
  "text_1": "A cat",
  "text_2": {
    "content": [
      {"type": "image_url", "image_url": {"url": "cat_img.jpg"}},
      {"type": "image_url", "image_url": {"url": "dog_img.jpg"}}
    ]
  }
}
```

### Offline Deployment

```python
from vllm import LLM

llm = LLM(
    model="jinaai/jina-reranker-m0",
    tensor_parallel_size=2,
    gpu_memory_utilization=0.75,
    max_model_len=1024,
    max_num_seqs=32,
    kv_cache_dtype="fp8",
    dtype="bfloat16",
)

res = llm.score("fast recipes for weeknight dinners", [
    "A 65-minute pasta with garlic and olive oil.",
    "Slow braised short ribs that cook for 5 hours.",
    "Stir-fry veggies with pre-cooked rice.",
])
for item in res:
    print(item.outputs.score)
```

### References

- [jina-reranker-m0 on Hugging Face](https://huggingface.co/jinaai/jina-reranker-m0)
- [vLLM Score API documentation](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#score-api)