CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

In 30 Sekunden

Was passiert ist

CordisBench, a 1,200-question benchmark, tests whether language models can reason about component lifecycles in dynamic agent systems that manage dependencies and cleanup.

Warum es zählt

Matters for engineers building AI agents that modify their own execution environment or manage complex plugin and dependency systems at runtime.

Achtung

Models degrade significantly as system complexity grows; reasoning costs are high, with GPT-5.6 Luna using nearly 3,000 tokens per question at medium effort.