教AI只学老师的排名，不学老师的自信
蒸馏大模型时，我们一直让小学徒去模仿老师的「打分」——每个词该得几分。但这份分数里混着两样东西：老师喜欢哪个词，以及老师有多自信。自信是老师自己的本事，小学徒学不来，还特别受一个叫「温度」的参数影响，调不好就翻车。这篇论文做了个大胆的减法：只学老师的「排名」，前几个候选词的先后顺序，分数本身扔掉。结果在六组师生模型上，数学推理成绩全面反超传统方法，最高涨5.6分。它不是你明天能用上的，但它指了个方向：大模型教小模型，真正传下去的是偏好顺序，不是自信程度。
📄 原文摘要(英文)
On-policy distillation (OPD) trains a student on its own generations by matching the teacher's token probabilities. These probabilities entangle two kinds of information: which tokens the teacher prefers, and how confident it is. The latter reflects the teacher's own capacity and calibration, which a smaller student may be unable to reproduce. Because the teacher temperature changes these values without changing the ordering, OPD is also sensitive to the temperature choice. We ask whether the teacher's ordering of candidate tokens is enough, and propose Order-Only On-Policy Distillation (O^{3}PD). At each student-generated prefix, O^{3}PD trains the student to reproduce the order of the teacher's top-K tokens, without fitting their probability values. The objective is therefore invariant to the teacher temperature and needs no temperature tuning. Across six teacher--student pairs, O^{3}PD outperforms OPD on the average over HMMT25, AIME24, and AIME25 in every pair, by up to 2.9 points, with gains of up to 5.6 points on AIME25. Other ranking losses yield similar gains, so the benefit comes from ordinal supervision itself. Analysis shows that O^{3}PD departs further from the teacher's probabilities than OPD, yet reproduces its ranking more faithfully and yields a stronger student. The ranking, not the probability values, thus carries the knowledge that on-policy distillation needs to transfer.