Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

-->

Computer Science > Artificial Intelligence

arXiv:2607.14004v1 (cs)

[Submitted on 15 Jul 2026]

Title: Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

Authors: Wenxiao Wang , Priyatham Kattakinda , Soheil Feizi

View a PDF of the paper titled Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0, by Wenxiao Wang and 2 other authors

View PDF HTML (experimental)

Abstract: Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method. This does not test the setting that matters for deployed agents, where optimization is applied recursively as new failures and new tasks appear over time. The central question this raises is whether optimizer-driven gains compound: after an agent has been optimized once, can it be optimized again on newly arrived tasks without eroding the gains the first round produced? We study this question with a two-phase continual-learning evaluation built from hard tasks in Terminal-Bench 2.0, comparing three approaches to agent-harness optimization (GEPA, Meta Harness, and RELAI's Verifiable Continual Learning, RELAI-VCL) under identical optimization budgets. All three methods improve over the baseline agent in the conventional, static, single-phase

…