正在加载模式…
⌨️
Terminal-Bench(TB)
Hard command-line tasks in isolated environments, each with a human-written solution and tests that decide whether the agent actually finished.
复杂度: high评估与监控
30秒速览
- 是什么
- Poses hard command line tasks in isolated containers, each carrying a human-written solution and tests that decide afterwards whether the work was actually finished.
- 何时使用
- You are evaluating agents that do infrastructure, build or debugging work in a shell, and you need correctness decided rather than judged.
- 注意
- The score names a model and a harness together: the same weights under a weaker scaffold give up exactly where a stronger one reads the error and recovers.
向AI专家咨询此模式
打开助手并预填您的问题,发送前可先确认。
Terminal-Bench: 概览
Hard command-line tasks in isolated environments, each with a human-written solution and tests that decide whether the agent actually finished.
- One container per task, so runs cannot contaminate each other
- Every task carries a human-written reference solution
- Verified by tests rather than by inspecting the transcript
- Drawn from real infrastructure, data and debugging work
- Deliberately hard: frontier systems remain well short of solving it
- Harness-agnostic, so the agent and the scaffold are measured together
The Agent Architect
每周一个模式、一个权衡、一个生产事故案例。为构建智能体系统的人准备的每周简报。
每周一封邮件,一键退订。您的地址仅用于发送简报。
又称： CLI agent benchmark, Command line agent evaluation
参考资料
该模式所依据的论文、规范和代码仓库。