训练AI智能体，别再让它从头跑到尾
训练一个能完成多步任务的AI智能体（比如帮你订机票、查资料），传统方法让它把整个任务从头跑到尾，再跟老师对比学习。但问题来了：任务后半段往往又慢又没信息量，AI学了半天只是在重复开头那几步。这篇论文提出一个“按步分配预算”的策略：先让AI试跑几步，根据表现决定要不要继续跑下去；同时调整学习重心，让AI在每一步都学到东西，而不是只盯着开头。实验证明，在同等训练时间下，AI的准确率更高。这不是你明天就能用的工具，但它指向一个趋势：训练AI要更聪明地分配时间，而不是一味堆算力。
📄 原文摘要(英文)
On-policy distillation (OPD) trains a student policy by matching a stronger teacher on the student's own trajectories, offering a promising framework for language agent training. However, its application to long-horizon agentic tasks remains insufficiently explored. We identify two key inefficiencies in vanilla agent OPD: (1) full-horizon rollouts often waste wall-clock resources on tail turns that provide weak and noisy KL supervision, and (2) trajectory-level KL objectives concentrate most of the loss on shallow tokens, leaving deeper decision turns under-trained once initial behaviors are aligned. To address these challenges, we propose TurnOPD, a turn-level budgeting strategy for efficient on-policy distillation of long-horizon agents. TurnOPD consists of two budget controllers: adaptive rollout-depth budgeting, which uses probe-based turn statistics to determine rollout length, and progressive turn-normalized loss budgeting, which gradually shifts KL weighting from token-level to turn-balanced supervision. Experiments on ALFWorld, WebShop, and Multi-Hop Search with task-specialized teacher models show that TurnOPD achieves superior validation accuracy under equal wall-clock training budgets and advances the accuracy--time frontier beyond vanilla OPD.