让AI变聪明的新招：故意学错，再绕开
让AI变强的主流思路是让它模仿正确答案。这篇论文反着来：先让AI扮演一个“粗心鬼”，故意生成一堆错误推理，然后训练它远离这些错误。难点在于，错误推理里混着正常的语言表达，一刀切地惩罚会把AI的基础语言能力也毁了。研究者设计了一个动态开关，只精准打击“推理错误”那部分，保住语言底子。在多个推理任务上，这种“反向学习”比正向模仿效果更好。它不是你明天就能用上的功能，但提示了一个趋势：AI自我进化不只有“学对”一条路，“避开错”同样有效。
📄 原文摘要(英文)
On-Policy Self-Distillation (OPSD) has emerged as a popular paradigm for large language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However, recent findings indicate that OPSD can severely degrade the performance of LLMs on complex reasoning tasks: By forcing the student to imitate an artificially confident reasoning trace conditioned on privileged information, OPSD inadvertently suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behaviors required to solve challenging problems. To address this, we introduce Negative Self-Distillation (NSD), a new framework that optimizes LLMs by diverging from flawed reasoning rather than imitating privileged solutions. Instead of relying on ground-truth answers or external supervision, NSD uses the model itself to generate a question-specific negative condition (eg, acting as a ``careless reasoner'') and pushes the student's distribution away from this self-generated negative teacher. Naively applying unlearning objectives to achieve this divergence is problematic, as flawed reasoning tokens are confounded with basic linguistic tokens; indiscriminately penalizing both risks catastrophically degrading the model's foundational language capabilities. We resolve this by designing a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens, ensuring gradient updates target only behavioral flaws while preserving the model's linguistic priors. Empirically, NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning (RL) baselines.