In the news
Making open models work for long-horizon agentic workloads
Together AI · Published · 3 min read
In 30 seconds
- What happened
- Together AI and Ollama documented how to build agentic applications using prefix caching to avoid reprocessing long conversation histories on every inference turn.
- Why it matters
- Engineers building coding agents or long-horizon agentic systems need this when optimizing inference cost and latency as session length grows.
- Watch out
- Prefix caching breaks if anything changes before new content in the prompt; even small changes near the top force reprocessing the entire session, negating cache benefits.
- agent
- agentic
- long-horizon
- llama
The patterns behind this
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.