vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B300

In 30 Sekunden

Was passiert ist

Novita AI open-sourced Chord, a W4A16 MoE CUDA operator for INT4 quantized weights, achieving 1.3x speedup on H200 and 2.15x on B300.

Warum es zählt

Matters for engineers serving Kimi K2.x models or other INT4 MoE systems on Hopper and Blackwell GPUs seeking inference performance gains.

Achtung

Grouped kernel integration with vLLM remains work-in-progress; indexed path is production-ready but requires explicit quantization humming selection to avoid automatic backend fallback.