StateBench: Testing how language models track state changes over long documents

In 30 Sekunden

Was passiert ist

StateBench benchmark tests how language models track entity state changes across long documents with four difficulty levels.

Warum es zählt

Matters for engineers building systems that must follow variable reassignments, code execution, or narrative state across extended contexts.

Achtung

Frontier models still fail on harder state tracking as inputs grow longer. Framework assumes fixed variables, not realistic scenarios where new entities appear.