AI蒸馏中的“特权幻觉”被破解
训练小模型时，通常让大模型（老师）指导小模型（学生）学习。但若老师或学生拥有额外信息（特权），学生可能误把“信息不对称”当作“能力差距”来模仿，导致学不会真本事——这叫“特权幻觉”。新方法DOPD能动态判断每个词该由老师教还是学生自己学，避免学生被虚假信号误导。实验证明，它在语言和视觉模型上均优于传统方法。虽然你明天用不上，但它让AI训练更高效、更可靠。
📄 原文摘要(英文)
On-policy distillation (OPD) offers superior capacity transfer by supervising student-sampled trajectories with dense token-level signals. To furnish high-quality supervision sources and thereby elevate the performance frontier of distillation, an intuitive direction is to infuse privileged information to either teacher or student itself. However, this additional input induces a potential failure mode we dub privilege illusion: a pattern that conflates the transferable capability gap that students are meant to close, and the information asymmetry gap that can only be mimicked but never replicated. This issue is further amplified by the inherent non-uniformity of token-level supervision, where only a small subset of tokens carries pivotal capability-bearing signals. To this end, we propose DOPD, an advantage-aware dual distillation paradigm that dynamically routes token-level supervision between privileged teacher and privileged student policies based on their advantage gap and relative probabilities. Each token receives supervision of different strength, objective, and strategy from either teacher or student itself, which transfers credible capability while simultaneously receiving auxiliary signals, to alleviate privilege illusion. Extensive experiments on both large language model (LLM) and vision-language model (VLM) settings demonstrate that DOPD consistently outperforms Vanilla OPD and other counterparts. Further results on stability, robustness, continual learning, and out-of-distribution tasks validate its superiority.