In the news
vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B300
vLLM · Novita AI and the vLLM Team · Published · 3 min read
In 30 seconds
- What happened
- Novita AI open-sourced Chord, a W4A16 MoE CUDA operator for INT4 quantized weights, achieving 1.3x speedup on H200 and 2.15x on B300.
- Why it matters
- Matters for engineers serving Kimi K2.x models or other INT4 MoE systems on Hopper and Blackwell GPUs seeking inference performance gains.
- Watch out
- Grouped kernel integration with vLLM remains work-in-progress; indexed path is production-ready but requires explicit quantization humming selection to avoid automatic backend fallback.
- llm
- kernel
- cuda
- serving
- vllm
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.