News Hub

vLLM migrated to LibTorch Stable ABI to enable PyTorch upgrades without rebuilding vLLM.

Asynchronous RL aligns rollout and training kernels with unclipped importance sampling for DeepSeek-V4.1-Flash and Kimi K3.1.

Kinference provides open source inference control plane for vLLM and Kubernetes with rate limiting and cost tracking.

vLLM supports Kimi K3.1 Pro with sparse attention, packed KV cache, and fused low-latency decode.

vLLM provides day-0 support for Mistral Large 4 with sparse MLA and speculative decoding.

vLLM serves GLM-5.3 on 16 H200s with PCP and DCP prefill, reaching 1M-token contexts and 1.7-3.2x per-user speed improvement.

vLLM provides day-0 support for MiniMax-M3.1-Flash on NVIDIA Blackwell, Rubin, and AMD MI350 GPUs.

vLLM serves DeepSeek-V4.1-Flash on GB300 NVL72 at 954K tokens per second per GPU.

vLLM supports Flint checkpoints with 2.5B active parameters, achieving 500 tokens/s on one GPU.

Ant Group and vLLM community released Humming, a quantized inference kernel library with broad GPU support.

vLLM TorchTPU is a new backend for Google Cloud TPUs with multi-host parallelism and KV cache offloading.

vLLM achieved near-zero-downtime scaling in Elastic Expert Parallelism using asynchronous preparation and NIXL EP.

Kimi K3 optimization on AMD Instinct MI355X matches NVIDIA HGX B300 throughput per GPU.

vLLM supports Reflection Beam-1.0 with 2.71x decode latency reduction and adaptive speculative decoding.

vLLM adds day-0 support for Muse Spark Open, an 816B MoE model with 1M context window and multimodal capabilities.

vLLM added day 0 support for NVIDIA Nemotron 3.5 Super VL with multimodal capabilities and multi-token prediction.

vLLM achieves bitwise-identical logprobs between rollout and training forward passes on specific models while maintaining 91% rollout throughput.

vLLM's push-based KV transfer improves time-to-first-token and time-per-output-token for disaggregated inference.

vLLM's adaptive verification uses online-fitted acceptance estimator without requiring confidence head.

vLLM-Omni runs MAGI-2 Preview, a 114B audio-video MoE model, on Moore Threads GPUs using torchada's CUDA compatibility layer.