vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B300

За 30 секунд

Что произошло

Novita AI open-sourced Chord, a W4A16 MoE CUDA operator for INT4 quantized weights, achieving 1.3x speedup on H200 and 2.15x on B300.

Почему это важно

Matters for engineers serving Kimi K2.x models or other INT4 MoE systems on Hopper and Blackwell GPUs seeking inference performance gains.

На что обратить внимание

Grouped kernel integration with vLLM remains work-in-progress; indexed path is production-ready but requires explicit quantization humming selection to avoid automatic backend fallback.