In the news
What We Learned by Reproducing 2,200 papers from ICML
Hugging Face · Published · 3 min read
In 30 seconds
- What happened
- A hackathon reproduced 2,226 ICML 2026 papers using coding agents, finding 51% had verified claims and 23% had falsified or contested claims.
- Why it matters
- Matters for researchers evaluating conference paper reliability and for those using AI agents to audit scientific work at scale.
- Watch out
- Reproducibility is adversarial, not binary. Independent teams reached opposite verdicts on same claims. Missing artifacts limited many reproductions.
Listen to this summary
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.