AI推理跑偏？让老师中途接手纠正
大模型在推理时，一旦开头想错方向，后面就会一路错到底，浪费算力。这篇论文发现，老师模型（更强的AI）在学生模型犯错时，会倾向于“转向”，而学生则“一条道走到黑”。研究者利用这个差异，设计了一个“接力训练”方法：在学生推理过程中，一旦检测到可能跑偏，就让老师模型短暂接管几步，给出正确方向，再交还学生继续。这样既纠正了错误，又保留了学生自己的推理风格。在8个数学推理测试中，该方法比传统方法平均提升5.73%，且训练轨迹长度减少50%以上。这不是你明天能直接用的工具，但它揭示了AI训练中一个被忽视的“纠错时机”——就像学数学时，老师在你刚写错第一步时就指出来，比等你算完再讲效率高得多。
📄 原文摘要(英文)
On-policy distillation (OPD) grounds token-level supervision in the student's own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher-student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label-free handoff trigger in Relay On-Policy Distillation (Relay-OPD). During training, Relay-OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qwen3-4B-Instruct-2507 teacher and Qwen3-0.6B/1.7B-Non-Thinking students on eight mathematical reasoning benchmarks, Relay-OPD achieves the best or second-best results on every benchmark, outperforming standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for 1.7B, with consistent gains at 0.6B. Training trajectory length is reduced by over 50%.