AI学会从失败中提炼“技能”，自己教自己
大模型做复杂任务时，常常走一步算一步，直到最后才知道成败。这篇论文让AI学会“复盘”：做完一个任务后，自己分析整个过程的轨迹，提炼成一条条自然语言写的“技能”，比如“先查文档再调用工具”或“遇到错误代码要回滚”。然后在下一次做类似任务时，这些技能会被当作额外的监督信号，在每一步都告诉模型“刚才那样选更好”。整个过程完全自动，不需要人类标注。在文本和视觉的长期任务测试中，这种自我蒸馏方法让成功率提升明显，且能泛化到没见过的场景。它不是你明天能用上的，但指向了一个方向：AI不再只靠试错，而是开始从自己的经验中抽象出规则，像人一样“吃一堑，长一智”。
📄 原文摘要(英文)
Large language models are increasingly trained as interactive agents for long-horizon tasks involving multi-turn interaction, tool use, and environment feedback. Outcome-based reinforcement learning (RL) provides a practical optimization paradigm, but its sparse trajectory-level rewards offer limited guidance on intermediate decisions, leaving a supervision gap between episode-level outcomes and token-level policy learning. We propose SEED (SElf-Evolving On-Policy Distillation), a self-evolving framework that converts completed on-policy trajectories into training-time hindsight skills and distills their behavioral effect back into the policy model. SEED first fine-tunes the policy to analyze completed trajectories and generate natural-language skills that capture reusable workflows, decisive observations, or failure-avoidance rules. During RL, the current policy both collects trajectories and serves as the analyzer that extracts hindsight skills from them. Policy updates therefore improve subsequent decision making and skill analysis together, allowing hindsight supervision to evolve with the policy. SEED then re-scores the sampled actions under ordinary and skill-augmented contexts, converting the skill-induced probability shift into a dense token-level on-policy distillation signal. This signal is jointly optimized with outcome-based RL, keeping the auxiliary supervision aligned with the current trajectory distribution. Extensive experiments on text-based and vision-based agentic tasks show that SEED consistently improves performance and sample efficiency, exhibiting robust generalization to unseen scenarios. Our code is available at https://github.com/jinyangwu/SEED.