In the news
Perplexity and Intelligence
OpenAI · Published · 3 min read · 100 on Hacker News
In 30 seconds
- What happened
- OpenAI proposes perplexity, a pretraining metric measuring prediction accuracy, as a shared industry standard for tracking AI progress and establishing safety pacing limits.
- Why it matters
- Matters for AI researchers and safety teams evaluating frontier models and coordinating development pace across competing labs.
- Watch out
- Perplexity doesn't directly measure real-world capabilities, varies across datasets and training data, and doesn't account for post-training improvements like reasoning.
- eval
The patterns behind this
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.