老师不教梯度，只改题目：小模型也能学得更好
传统知识蒸馏让大模型当老师，小模型模仿它的输出，但老师太强时，小模型反而学偏——只记住老师最尖锐的答案，在没见过的问题上表现差。这篇论文换了个思路：老师不直接教答案，而是把难题改写成两种选择题。一种把老师对的和小模型错的混在一起，让小模型分辨；另一种把小模型犯过的错集中展示，让它看清自己哪里不行。这样小模型在自己能理解的范围内反复练，直到正确率过半才算毕业。在31个测试集上，小到0.8B参数的小模型，效果比传统蒸馏和强化学习都好。它不是你明天能用上的，但给了一个新方向：教AI不一定靠灌输，可以靠设计更好的题目。
📄 原文摘要(英文)
Knowledge distillation transfers a teacher's competence to a small student but is brittle in the small-student regime: forcing the student to imitate logits from a much larger teacher concentrates it on the teacher's sharpest modes, hurting generalization on benchmark families beyond the training corpus. Reinforcement learning (RL) avoids logit imitation by training on the student's own rollouts. However, on questions where every rollout fails-yielding zero advantage and being silently discarded-injecting a stronger teacher's response into the policy gradient breaks the on-policy assumption and induces drift. We introduce Zone of Proximal Policy Optimization (ZPPO), inspired by Vygotsky's zone of proximal development, which keeps the teacher inside the prompt rather than the policy gradient. On hard questions, ZPPO constructs two reformulated prompts: a Binary Candidate-included Question (BCQ) pairs one correct teacher response with one incorrect student response as anonymized candidates the student must discriminate, and a Negative Candidate-included Question (NCQ) aggregates the student's wrong rollouts into a single prompt to surface their shared failure modes. A prompt replay buffer recirculates each hard question until it either graduates-the student's mean rollout accuracy on it reaches half- or is FIFO-evicted under finite capacity, amplifying BCQ and NCQ inside the student's current zone of proximal development. On the Qwen3.5 family at four student scales (0.8B-9B) with a 27B teacher, post-trained as vision-language models and evaluated on a 31-benchmark suite (16 VLM, 10 LLM, 5 Video), ZPPO outperforms off/on-policy distillation and GRPO, with the largest gains at the smallest scale.