AI 终于学会说“我不会”了
大模型在超出能力的问题上会硬着头皮编答案，生成一大段听起来合理、其实全错的推理，既浪费算力又误导人。这篇研究把这种现象拆开看，发现模型最大的毛病是“貌似有理的胡扯”——输出表面光鲜，错误藏在细节里，而且任务越难越严重。他们提出的 CaRL 方法，用强化学习给模型一个选择：没把握就拒绝回答，而不是硬编。结果胡扯大幅减少，该会的题照样会。它不是你明天能用上的工具，但这是 AI 从“不懂装懂”走向“知道自己不知道”的关键一步。
📄 原文摘要(英文)
Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks, creating risks where plausible-sounding but incorrect derivations mislead users. We characterize this futile reasoning phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce CaRL (Capability-aligned Reinforcement Learning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability-aligned behavior without sacrificing utility. https://github.com/icip-cas/Knowing-When-to-Quit