新闻
Pass the Baton: Trajectory-Relayed On-Policy Distillation
arXiv cs.AI · 发布于 · 阅读约3分钟
30秒读懂
- 发生了什么
- Relay-OPD improves student model training by having a teacher model take over when the student commits to wrong reasoning directions, then resuming student training on corrected trajectories.
- 为何重要
- Matters for engineers training smaller language models on reasoning tasks who want better performance with less compute spent on misdirected generations.
- 注意
- Method requires detecting failure points and managing teacher intervention budget; unclear how well trigger detection generalizes across different model sizes and task domains.
收听本摘要
文章节选
-->
Computer Science > Computation and Language
arXiv:2607.26057v1 (cs)
[Submitted on 28 Jul 2026]
Title: Pass the Baton: Trajectory-Relayed On-Policy Distillation
Authors: Haolei Xu , Xiaowen Xu , Haiwen Hong , Zixuan Ni , Hongxing Li , Yiwen Qiu , Weiming Lu , Yongliang Shen
View a PDF of the paper titled Pass the Baton: Trajectory-Relayed On-Policy Distillation, by Haolei Xu and 7 other authors
View PDF HTML (experimental)
Abstract: On-policy distillation (OPD) grounds token-level supervision in the student's own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher-student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label-free handoff trigger in Relay On-Policy Distillation (Relay-OPD). During training, Relay-OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qw
节选自原文。请前往来源阅读全文。
- reasoning
The Agent Architect
每周一个模式、一个权衡、一个生产事故案例。为构建智能体系统的人准备的每周简报。
每周一封邮件,一键退订。您的地址仅用于发送简报。