Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR

30秒で要点

何が起きたか

Researchers show that training language models with on-policy distillation first, then reinforcement learning with verifiable rewards, outperforms joint optimization on reasoning tasks.

なぜ重要か

Engineers building post-training pipelines for reasoning models should consider this sequential approach instead of combining both signals in a single training step.

注意点

The paper tests only logic and math reasoning benchmarks; effectiveness on other domains and the practical overhead of two-stage training remain unclear.