不给老师，AI 自己当自己的教练
训练生成模型一向要两步：先请个「老师」模型把标准答案蒸馏给学生，再让学生模仿。这篇把老师直接裁了——学生每走一步，自己先多走几条岔路，看哪条走岔了、哪条走顺了，用这个差异当反馈把自己拉回正路。同一个模型，既当选手又当裁判，训练成本砍掉一大截，而且效果还超过了原来请老师的那些方法。它不是让你明天用上，而是提示一个趋势：AI 可能不再需要标杆，自己试错就能变强。
📄 原文摘要(英文)
On-policy distillation (OPD), which leverages a pre-trained, specialized teacher model to provide dense supervisory signals, has achieved significant success in Large Language Models (LLMs) and has recently been adapted to flow matching models. However, this paradigm suffers from two major issues: First, training a separate, task-specific teacher for every new objective incurs high computational costs. Second, the discrepancy between teacher and student distributions often leads to compounding errors along the generation trajectory. In this paper, we introduce Self-OPD, a teacher-free OPD framework for flow matching models that turns the student's own self-exploration into step-wise supervision. At each timestep, Self-OPD branches the deterministic next-state prediction into K stochastic SDE candidates, rolls them out with the ODE sampler, and compares their rewards against a deterministic self-reference baseline to obtain normalized advantages. The velocity field is optimized with an all-branch pull-push objective, where high-advantage branches attract the student and low-advantage branches repel it under direction-aware attenuation and SDE-variance normalization. For multi-objective alignment, Self-OPD fuses normalized scores at the reward level, avoiding direct gradient conflict. Experiments on single and mixed reward benchmarks show that Self-OPD outperforms prior RL and OPD methods without task-specific teachers.