新闻
Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0
arXiv cs.AI · 发布于 · 阅读约3分钟
30秒读懂
- 发生了什么
- Researchers tested whether agent optimization methods maintain gains when applied repeatedly to new tasks over time using Terminal-Bench 2.0.
- 为何重要
- Matters for engineers deploying agents in production where continuous optimization happens as new failures and tasks emerge.
- 注意
- Only RELAI-VCL maintained and improved gains across optimization rounds; other methods either degraded or plateaued, suggesting most gains may not be stable.
收听本摘要
文章节选
-->
Computer Science > Artificial Intelligence
arXiv:2607.14004v1 (cs)
[Submitted on 15 Jul 2026]
Title: Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0
Authors: Wenxiao Wang , Priyatham Kattakinda , Soheil Feizi
View a PDF of the paper titled Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0, by Wenxiao Wang and 2 other authors
View PDF HTML (experimental)
Abstract: Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method. This does not test the setting that matters for deployed agents, where optimization is applied recursively as new failures and new tasks appear over time. The central question this raises is whether optimizer-driven gains compound: after an agent has been optimized once, can it be optimized again on newly arrived tasks without eroding the gains the first round produced? We study this question with a two-phase continual-learning evaluation built from hard tasks in Terminal-Bench 2.0, comparing three approaches to agent-harness optimization (GEPA, Meta Harness, and RELAI's Verifiable Continual Learning, RELAI-VCL) under identical optimization budgets. All three methods improve over the baseline agent in the conventional, static, single-phase
节选自原文。请前往来源阅读全文。
- agent
The Agent Architect
每周一个模式、一个权衡、一个生产事故案例。为构建智能体系统的人准备的每周简报。
每周一封邮件,一键退订。您的地址仅用于发送简报。