发生了什么: Inkling 2 Small on vLLM: Day-0 Support

vLLM added day-0 support for Inkling 2 Small, a 594B parameter multimodal MoE model with 1M token context and native speculative decoding.

为何重要: Inkling 2 Small on vLLM: Day-0 Support

Engineers deploying large multimodal models who need high throughput inference on multi-GPU clusters with long context windows.

注意: Inkling 2 Small on vLLM: Day-0 Support

Model requires tensor parallelism across multiple GPUs; performance scales significantly with context length, dropping from 1,070 to 450 tokens per second as context grows from 32K to 256K.

新闻

Inkling 2 Small on vLLM: Day-0 Support

vLLM · Inferact and Thinking Machines · 发布于 2026年11月12日 · 阅读约3分钟

30秒读懂

发生了什么: vLLM added day-0 support for Inkling 2 Small, a 594B parameter multimodal MoE model with 1M token context and native speculative decoding.
为何重要: Engineers deploying large multimodal models who need high throughput inference on multi-GPU clusters with long context windows.
注意: Model requires tensor parallelism across multiple GPUs; performance scales significantly with context length, dropping from 1,070 to 450 tokens per second as context grows from 32K to 256K.

阅读完整文章

llm
attention
speculative
kernel
vllm

这条新闻背后的模式

每个模式都讲清楚技术如何运作、何时值得投入，以及在哪里会失效。

The Agent Architect

每周一个模式、一个权衡、一个生产事故案例。为构建智能体系统的人准备的每周简报。

每周一封邮件,一键退订。您的地址仅用于发送简报。