{ "hf_id": "jinaai/jina-reranker-m0", "meta": { "title": "Jina Reranker m0", "slug": "jina-reranker-m0", "provider": "Jina AI", "description": "Multilingual, multimodal reranker for text and visual documents across 29+ languages via Qwen2-VL backbone", "date_added": "2025-10-29", "date_updated": "2026-04-17", "tasks": [ "embedding" ], "related_recipes": [] }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve jinaai/jina-reranker-m0 \\\n --gpu-memory-utilization 0.75 \\\n --max-num-seqs 32 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "jinaai/jina-reranker-m0", "--gpu-memory-utilization", "0.75", "--max-num-seqs", "32", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest jinaai/jina-reranker-m0 \\\n --gpu-memory-utilization 0.75 \\\n --max-num-seqs 32 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "jinaai/jina-reranker-m0", "--gpu-memory-utilization", "0.75", "--max-num-seqs", "32", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/jinaai/jina-reranker-m0/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/jinaai/jina-reranker-m0/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/jinaai/jina-reranker-m0/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/jinaai/jina-reranker-m0/hw/h200.json", "h100": "/jinaai/jina-reranker-m0/hw/h100.json", "b200": "/jinaai/jina-reranker-m0/hw/b200.json", "gb200": "/jinaai/jina-reranker-m0/hw/gb200.json", "b300": "/jinaai/jina-reranker-m0/hw/b300.json", "gb300": "/jinaai/jina-reranker-m0/hw/gb300.json", "mi300x": "/jinaai/jina-reranker-m0/hw/mi300x.json", "mi325x": "/jinaai/jina-reranker-m0/hw/mi325x.json", "mi350x": "/jinaai/jina-reranker-m0/hw/mi350x.json", "mi355x": "/jinaai/jina-reranker-m0/hw/mi355x.json" } }, "model": { "model_id": "jinaai/jina-reranker-m0", "min_vllm_version": "0.10.0", "architecture": "dense", "parameter_count": "2.4B", "active_parameters": "2.4B", "context_length": 32768, "base_args": [ "--gpu-memory-utilization", "0.75", "--max-num-seqs", "32" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": {}, "opt_in_features": [], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 6, "description": "BF16 weights; 2x T4 or 2x L4 GPUs" } }, "hardware_overrides": { "amd": { "extra_args": [], "extra_env": { "VLLM_ROCM_USE_AITER": "1" } } }, "guide": "## Overview\n\n[jinaai/jina-reranker-m0](https://huggingface.co/jinaai/jina-reranker-m0) is a\nmultilingual, multimodal reranker that ranks visual documents across 29+ languages.\nIt accepts text and visual content, including pages with mixed text, figures, tables,\nand various layouts.\n\nDeployment target: 2x NVIDIA T4 or 2x NVIDIA L4.\n\n## Prerequisites\n\n- Hardware: 2x T4 or 2x L4 (or any 2x GPU with ~16 GB each)\n- vLLM >= 0.10.0\n\n### Install vLLM (CUDA)\n\n```bash\nuv pip install vllm\n```\n\n### Install vLLM (AMD ROCm)\n\n```bash\nuv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.14.1/rocm700\n```\n\n## Launch command\n\n```bash\nvllm serve jinaai/jina-reranker-m0 \\\n --host 0.0.0.0 \\\n --port 8000 \\\n --tensor_parallel_size 2 \\\n --gpu-memory-utilization 0.75 \\\n --max-num-seqs 32\n```\n\nOn AMD:\n\n```bash\nexport VLLM_ROCM_USE_AITER=1\nvllm serve jinaai/jina-reranker-m0 \\\n --tensor_parallel_size 2 --gpu-memory-utilization 0.75 --max-num-seqs 32\n```\n\n## Rerank API\n\n```bash\ncurl -X POST http://localhost:8000/v1/rerank \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"jinaai/jina-reranker-m0\",\n \"query\": \"What are the health benefits of green tea?\",\n \"documents\": [\n \"Green tea contains antioxidants called catechins...\",\n \"El precio del café ha aumentado un 20% este año...\",\n \"Studies show that drinking green tea regularly...\"\n ],\n \"top_n\": 3,\n \"return_documents\": true\n }'\n```\n\n## Score API\n\nText-to-text:\n\n```bash\ncurl -X POST http://localhost:8000/v1/score \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"jinaai/jina-reranker-m0\",\n \"text_1\": [\"What is the capital of Brazil?\"],\n \"text_2\": [\"The capital of Brazil is Brasilia.\"]\n }'\n```\n\nMultimodal (text vs. images):\n\n```json\n{\n \"model\": \"jinaai/jina-reranker-m0\",\n \"text_1\": \"A cat\",\n \"text_2\": {\n \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"cat_img.jpg\"}},\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"dog_img.jpg\"}}\n ]\n }\n}\n```\n\n## Offline Deployment\n\n```python\nfrom vllm import LLM\n\nllm = LLM(\n model=\"jinaai/jina-reranker-m0\",\n tensor_parallel_size=2,\n gpu_memory_utilization=0.75,\n max_model_len=1024,\n max_num_seqs=32,\n kv_cache_dtype=\"fp8\",\n dtype=\"bfloat16\",\n)\n\nres = llm.score(\"fast recipes for weeknight dinners\", [\n \"A 65-minute pasta with garlic and olive oil.\",\n \"Slow braised short ribs that cook for 5 hours.\",\n \"Stir-fry veggies with pre-cooked rice.\",\n])\nfor item in res:\n print(item.outputs.score)\n```\n\n## References\n\n- [jina-reranker-m0 on Hugging Face](https://huggingface.co/jinaai/jina-reranker-m0)\n- [vLLM Score API documentation](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#score-api)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "389ffebe555705c85337025d5df108d0384d365e", "path": "recipes/models/jinaai/jina-reranker-m0.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/jinaai/jina-reranker-m0.yaml", "overlays": [] } }