{
  "hf_id": "stepfun-ai/Step-5-Preview",
  "meta": {
    "title": "Step-5-Preview",
    "slug": "step-5-preview",
    "provider": "StepFun",
    "description": "Multimodal MoE with hybrid sparse/sliding-window attention, a 1M-token context window, and MTP.",
    "date_added": "2026-10-15",
    "date_updated": "2026-10-15",
    "tasks": [
      "multimodal"
    ],
    "default_hardware": "b300",
    "related_recipes": [
      "stepfun-ai/Step-3.7-Flash"
    ],
    "hardware": {
      "h200": "verified",
      "b300": "verified",
      "gb300": "verified"
    }
  },
  "recommended_command": {
    "hardware": "b300",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:step-5-preview",
    "command": "vllm serve stepfun-ai/Step-5-Preview \\\n  --data-parallel-backend mp \\\n  --distributed-executor-backend mp \\\n  --disable-custom-all-reduce \\\n  --max-model-len 1048576 \\\n  --max-num-batched-tokens 16384 \\\n  --max-num-seqs 64 \\\n  --gpu-memory-utilization 0.85 \\\n  --enable-prefix-caching \\\n  --tensor-parallel-size 8 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser step5 \\\n  --reasoning-parser step5 \\\n  --spec-method mtp \\\n  --spec-tokens 3",
    "argv": [
      "vllm",
      "serve",
      "stepfun-ai/Step-5-Preview",
      "--data-parallel-backend",
      "mp",
      "--distributed-executor-backend",
      "mp",
      "--disable-custom-all-reduce",
      "--max-model-len",
      "1048576",
      "--max-num-batched-tokens",
      "16384",
      "--max-num-seqs",
      "64",
      "--gpu-memory-utilization",
      "0.85",
      "--enable-prefix-caching",
      "--tensor-parallel-size",
      "8",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "step5",
      "--reasoning-parser",
      "step5",
      "--spec-method",
      "mtp",
      "--spec-tokens",
      "3"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:step-5-preview stepfun-ai/Step-5-Preview \\\n  --data-parallel-backend mp \\\n  --distributed-executor-backend mp \\\n  --disable-custom-all-reduce \\\n  --max-model-len 1048576 \\\n  --max-num-batched-tokens 16384 \\\n  --max-num-seqs 64 \\\n  --gpu-memory-utilization 0.85 \\\n  --enable-prefix-caching \\\n  --tensor-parallel-size 8 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser step5 \\\n  --reasoning-parser step5 \\\n  --spec-method mtp \\\n  --spec-tokens 3",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:step-5-preview",
      "stepfun-ai/Step-5-Preview",
      "--data-parallel-backend",
      "mp",
      "--distributed-executor-backend",
      "mp",
      "--disable-custom-all-reduce",
      "--max-model-len",
      "1048576",
      "--max-num-batched-tokens",
      "16384",
      "--max-num-seqs",
      "64",
      "--gpu-memory-utilization",
      "0.85",
      "--enable-prefix-caching",
      "--tensor-parallel-size",
      "8",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "step5",
      "--reasoning-parser",
      "step5",
      "--spec-method",
      "mtp",
      "--spec-tokens",
      "3"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "blackwell",
      "compute_arch": "sm103",
      "display_name": "B300",
      "description": "NVIDIA B300 SXM 268 GB HBM3e · 8-GPU HGX B300 node (Blackwell Ultra)",
      "gpu_count": 8,
      "vram_gb": 2144,
      "multi_node": false
    },
    "alternatives": {
      "single_node_tep": "/stepfun-ai/Step-5-Preview/strategies/single_node_tep.json",
      "single_node_dep": "/stepfun-ai/Step-5-Preview/strategies/single_node_dep.json",
      "multi_node_tp": "/stepfun-ai/Step-5-Preview/strategies/multi_node_tp.json",
      "multi_node_tep": "/stepfun-ai/Step-5-Preview/strategies/multi_node_tep.json",
      "multi_node_dep": "/stepfun-ai/Step-5-Preview/strategies/multi_node_dep.json",
      "pd_cluster": "/stepfun-ai/Step-5-Preview/strategies/pd_cluster.json",
      "kv_store_centralized_mooncake": "/stepfun-ai/Step-5-Preview/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/stepfun-ai/Step-5-Preview/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "b300": "/stepfun-ai/Step-5-Preview/hw/b300.json",
      "h100": "/stepfun-ai/Step-5-Preview/hw/h100.json",
      "h200": "/stepfun-ai/Step-5-Preview/hw/h200.json",
      "b200": "/stepfun-ai/Step-5-Preview/hw/b200.json",
      "gb200": "/stepfun-ai/Step-5-Preview/hw/gb200.json",
      "gb300": "/stepfun-ai/Step-5-Preview/hw/gb300.json",
      "mi300x": "/stepfun-ai/Step-5-Preview/hw/mi300x.json",
      "mi325x": "/stepfun-ai/Step-5-Preview/hw/mi325x.json",
      "mi350x": "/stepfun-ai/Step-5-Preview/hw/mi350x.json",
      "mi355x": "/stepfun-ai/Step-5-Preview/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "stepfun-ai/Step-5-Preview",
    "docker_image": "vllm/vllm-openai:step-5-preview",
    "architecture": "moe",
    "parameter_count": "604B",
    "context_length": 1048576,
    "base_args": [
      "--data-parallel-backend",
      "mp",
      "--distributed-executor-backend",
      "mp",
      "--disable-custom-all-reduce",
      "--max-model-len",
      "1048576",
      "--max-num-batched-tokens",
      "16384",
      "--max-num-seqs",
      "64",
      "--gpu-memory-utilization",
      "0.85",
      "--enable-prefix-caching"
    ],
    "base_env": {},
    "install": {
      "docker": {
        "command": "docker pull vllm/vllm-openai:step-5-preview",
        "note": "Use the dedicated Step-5 image for these launch commands."
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "Step-5 tool calling with automatic tool choice.",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "step5"
      ]
    },
    "reasoning": {
      "description": "Step-5 reasoning parser.",
      "args": [
        "--reasoning-parser",
        "step5"
      ]
    },
    "spec_decoding": {
      "description": "MTP with three speculative tokens.",
      "args": [
        "--spec-method",
        "mtp",
        "--spec-tokens",
        "3"
      ]
    }
  },
  "variants": {
    "default": {
      "precision": "fp8",
      "vram_minimum_gb": 773
    },
    "nvfp4": {
      "model_id": "stepfun-ai/Step-5-Preview-NVFP4",
      "precision": "nvfp4",
      "vram_minimum_gb": 473,
      "json": "/stepfun-ai/Step-5-Preview-NVFP4.json"
    }
  },
  "kv_offload_support": {
    "simple": "unsupported",
    "lmcache": "unsupported"
  },
  "strategy_min_gpus": {
    "gb200": 8,
    "gb300": 8,
    "h100": 16
  },
  "hardware_overrides": {
    "blackwell": {
      "extra_args": [
        "--moe-backend",
        "deep_gemm_mega_moe"
      ]
    }
  },
  "guide": "## Overview\n\n[Step-5 Preview](https://huggingface.co/stepfun-ai/Step-5-Preview) is a multimodal\nMoE model with hybrid sparse/sliding-window attention, a 1M-token context window,\nand multi-token prediction (MTP).\n\nThe [NVFP4 checkpoint](https://huggingface.co/stepfun-ai/Step-5-Preview-NVFP4)\nis available through the NVFP4 variant below.\n\n## Docker\n\n```bash\ndocker pull vllm/vllm-openai:step-5-preview\n\ndocker run --rm -it \\\n    --gpus all \\\n    --ipc=host \\\n    --network=host \\\n    --volume \"$HOME/.cache/huggingface:/root/.cache/huggingface\" \\\n    --entrypoint /bin/bash \\\n    vllm/vllm-openai:step-5-preview\n```\n\nRun the following commands inside the container. Model access is required;\nuse `hf auth login` if the checkpoint requires authentication.\n\n## FP8: single node on 4 B300 GPUs\n\nThe examples use four B300 GPUs. Choose TP4, TEP4, or DEP4:\n\n| Mode | Tensor parallel | Data parallel | Expert parallel |\n| --- | --- | --- | --- |\n| TP4 | 4 | 1 | Disabled |\n| TEP4 | 4 | 1 | 4 |\n| DEP4 | 1 | 4 | 4 |\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0,1,2,3\n```\n\nOn Blackwell, TEP4 and DEP4 use MegaMoE. TP4 uses the default MoE backend\nbecause MegaMoE requires expert parallelism.\n\nFor FP8 on H200, use eight GPUs: TP8, TEP8, or DEP8, respectively.\nSet `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7` and change the parallel size\nfrom 4 to 8 (DEP keeps TP=1). Omit the MegaMoE flag on Hopper.\n\nThese are separate layouts: TP4 and TEP4 use DP=1; DEP4 uses TP=1.\n`--max-num-seqs` applies per DP rank. The 1M-token context limit includes\ninput and output; reduce `--max-model-len` or concurrency if KV cache memory\nis insufficient.\n\n## NVFP4: single node on 4 B300 GPUs\n\nUse [stepfun-ai/Step-5-Preview-NVFP4](https://huggingface.co/stepfun-ai/Step-5-Preview-NVFP4)\nwith the same Step-5 image. The checkpoint includes both NVFP4 and BF16 layers;\nquantization is detected from its config.\n\n```bash\ndocker pull vllm/vllm-openai:step-5-preview\n\ndocker run --rm -it \\\n    --gpus 4 \\\n    --ipc=host \\\n    --network=host \\\n    --volume \"$HOME/.cache/huggingface:/root/.cache/huggingface\" \\\n    --entrypoint /bin/bash \\\n    vllm/vllm-openai:step-5-preview\n```\n\nInside the container, reuse one of the TP4, TEP4, or DEP4 commands above,\nreplacing the checkpoint with `stepfun-ai/Step-5-Preview-NVFP4` and setting\n`--max-model-len 320000`.\nKeep `--reasoning-parser step5` and `--tool-call-parser step5` unchanged.\n\n| Mode | Parallel arguments |\n| --- | --- |\n| TP4 | `--tensor-parallel-size 4` |\n| TEP4 | `--tensor-parallel-size 4 --enable-expert-parallel --moe-backend deep_gemm_mega_moe` |\n| DEP4 | `--tensor-parallel-size 1 --data-parallel-size 4 --enable-expert-parallel --moe-backend deep_gemm_mega_moe` |\n\nTP4 uses automatic MoE backend selection for the mixed-precision layers.\nTEP4 and DEP4 use MegaMoE for routed experts; shared experts run separately.\nSend requests with `\"model\": \"stepfun-ai/Step-5-Preview-NVFP4\"`.\n\nAvailable context and concurrency depend on the remaining KV cache memory.\n\n## Send a request\n\n```bash\ncurl http://127.0.0.1:8000/v1/chat/completions \\\n    --header 'Content-Type: application/json' \\\n    --data '{\n      \"model\": \"stepfun-ai/Step-5-Preview\",\n      \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n      \"max_tokens\": 4096,\n      \"stream\": true\n    }'\n```\n\n## References\n\n- [Step-5 Preview NVFP4 model](https://huggingface.co/stepfun-ai/Step-5-Preview-NVFP4)\n- [Step-5 Preview model](https://huggingface.co/stepfun-ai/Step-5-Preview)\n- [Step-5 Docker image](https://hub.docker.com/r/vllm/vllm-openai/tags?name=step-5-preview)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "389ffebe555705c85337025d5df108d0384d365e",
    "path": "recipes/models/stepfun-ai/Step-5-Preview.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/stepfun-ai/Step-5-Preview.yaml",
    "overlays": []
  }
}