In the news
Inkling 2 Small on vLLM: Day-0 Support
vLLM · Inferact and Thinking Machines · Published · 3 min read
In 30 seconds
- What happened
- vLLM added day-0 support for Inkling 2 Small, a 594B parameter multimodal MoE model with 1M token context and native speculative decoding.
- Why it matters
- Engineers deploying large multimodal models who need high throughput inference on multi-GPU clusters with long context windows.
- Watch out
- Model requires tensor parallelism across multiple GPUs; performance scales significantly with context length, dropping from 1,070 to 450 tokens per second as context grows from 32K to 256K.
- llm
- attention
- speculative
- kernel
- vllm
The patterns behind this
- Speculative & Parallel Tool Execution
- Agentic Context Engineering (Evolving Playbook)
- Context Window Management UI
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.