In the news
DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s
vLLM · Inferact and the vLLM Team · Published · 3 min read
In 30 seconds
- What happened
- vLLM achieved 58x higher throughput serving DeepSeek-V4.1-Flash on NVIDIA Vera Rubin NVL72 compared to GB300, reaching 1.7M tokens per second per GPU.
- Why it matters
- Matters for engineers deploying large language models who need maximum inference throughput and latency on latest NVIDIA hardware with specialized tensor cores.
- Watch out
- Results are specific to DeepSeek-V4.1-Flash model and SemiAnalysis AgentX benchmark; gains depend on Rubin's split memory domains and FP4 tensor cores, may not generalize to other models.
- agent
- llm
- throughput
- token
- vllm
The patterns behind this
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.