新闻
vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B300
vLLM · Novita AI and the vLLM Team · 发布于 · 阅读约3分钟
30秒读懂
- 发生了什么
- Novita AI open-sourced Chord, a W4A16 MoE CUDA operator for INT4 quantized weights, achieving 1.3x speedup on H200 and 2.15x on B300.
- 为何重要
- Matters for engineers serving Kimi K2.x models or other INT4 MoE systems on Hopper and Blackwell GPUs seeking inference performance gains.
- 注意
- Grouped kernel integration with vLLM remains work-in-progress; indexed path is production-ready but requires explicit quantization humming selection to avoid automatic backend fallback.
- llm
- kernel
- cuda
- serving
- vllm
The Agent Architect
每周一个模式、一个权衡、一个生产事故案例。为构建智能体系统的人准备的每周简报。
每周一封邮件,一键退订。您的地址仅用于发送简报。