一条训练数据就够？大模型蒸馏的惊人发现
训练大模型时，通常认为数据越多越好。但这项研究揭示了一个反直觉的事实：在一种叫「在线蒸馏」的训练方法中，只用一条查询数据，模型就能持续进步数百步，并恢复完整数据集训练效果的大部分。研究者发现，一条查询就能覆盖完整训练所触及状态的71.5%，而16条语义不同的查询就能达到98.9%，与全量数据训练持平。问题不在数据量，而在算法吸收速度——模型从教师那里获取监督的速度越来越慢。这就像给一个饥饿的人端上一桌菜，他第一口吃得很快，但消化速度跟不上。这项研究提醒我们，前沿AI训练的效率瓶颈可能不在数据，而在算法本身。它不会让你明天就用上，但它可能改变未来大模型训练的方式，让训练更省资源、更高效。
📄 原文摘要(英文)
On-policy distillation (OPD) combines student-generated rollouts with dense token-level supervision from a teacher. Existing work has mainly studied its algorithmic behavior, leaving the role of training data unclear. We examine this role at the data-minimal limit by training on a single query. One-shot OPD keeps improving for hundreds of steps and recovers most of full-data OPD's gain across task domains and model families. We explain this result through the states visited during training and the rate at which the student aligns with the teacher. We measure state coverage, the fraction of the states full-data OPD visits that a query set's rollouts reach. A single query already reaches \(71.5\%\), most of it within the first 100 steps. Adding semantically distinct queries raises coverage and validation accuracy together, until 16 queries reach \(98.9\%\) and match full-data training. Yet alignment slows at a similar pace whether OPD trains on one query or the whole dataset, and even a fixed set of states takes hundreds of steps to absorb. OPD is therefore data-overfed but algorithm-starved. Its rollouts quickly expose broad supervision, while the student absorbs that supervision increasingly slowly. The state-coverage result extends to multi-teacher OPD, where 16 semantically diverse queries per domain match full-data MOPD. As a further stress test, content-light templates and off-domain WildChat queries also approach the real-query baseline. Task content and induced state coverage can therefore come apart. We hope these findings direct future work toward the step efficiency of OPD, and prompt a re-examination of the data and the mechanisms behind its recent successes in frontier post-training.