AI自己教自己：用解码轨迹提升推理
大模型通常靠一步步预测下一个词来生成文本，但有一种新模型叫扩散语言模型，它像图片生成一样，从一堆乱码中逐步去噪得到完整句子。这种模型能并行生成，速度更快，但很难通过后训练提升推理能力——监督学习会暴露偏差，强化学习又太稀疏。研究者提出一种新方法：让模型自己当自己的老师，但不是用外部正确答案，而是利用模型自身在去噪过程中更靠后的、更清晰的解码步骤来指导前面的步骤。在数学推理和代码生成任务上，这种方法比传统监督学习和强化学习都强。它不是你明天就能用上的，但它展示了一种让AI自我改进的新思路：从自己的生成过程中学习，而不是依赖外部标签。
📄 原文摘要(英文)
Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods. On-policy self-distillation (OPSD) offers a promising alternative, using one model as both student and teacher to provide dense, token-level, on-policy supervision, but its effectiveness hinges on giving the teacher privileged information (PI) - typically an instance-specific ground-truth reference unavailable at inference - so the student ends up distilling a weak PI-free consensus policy that yields little improvement on dLLM reasoning. We introduce dOPSD, which instead derives the teacher's privilege directly from the student's own denoising trajectory, evaluating masked positions using later, more-decoded steps of that same trajectory rather than an external label, so the teacher's advantage emerges from the model's own decoding process; on Dream and LLaDA, dOPSD improves both in-domain math reasoning and out-of-domain code generation, outperforming supervised and on-policy baselines.