SocietyBench: Forecasting Counterfactual Social-World Evolution

In 30 Sekunden

Was passiert ist

SocietyBench is a benchmark that measures how well large language models forecast social events by predicting outcomes from anonymized, counterfactual timelines built from real news and social media.

Warum es zählt

Matters for engineers building LLM agents that need to understand and predict real-world social dynamics beyond task completion like bug fixing or browser control.

Achtung

Top models score only 75 out of 100 against a 50-point baseline. Performance splits into two independent axes: probability calibration and temporal accuracy, meaning strength in one does not guarantee the other.