In the news
Last year we built megakernels. This year we deleted them: The road to Together Compiler
Together AI · Published · 3 min read
In 30 seconds
- What happened
- Together AI built megakernels for AI inference last year but found they were impractical. This year they achieved 98% of megakernel performance using standard tools instead.
- Why it matters
- Engineers optimizing AI model inference on GPUs should care when balancing performance gains against implementation complexity and production maintainability.
- Watch out
- The 98% recovery uses CUDA Graphs and PDL on Torch Inductor kernels, not raw megakernels. Results are specific to Llama 1B on H100; generalization to other models and hardware remains unclear.
- agent
- kernel
- token
- llama
The patterns behind this
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.