让AI多算几步就能解更长的题，秘诀是教它每一步该走多远
循环语言模型理论上能多跑几轮，但多跑不等于能解更长的题——就像让一个人反复读题，不告诉他每遍该推进到哪，读一百遍也白搭。这篇论文给了一个具体解法：训练时让模型在第k轮迭代时，预测的正好是前k步操作之后的状态，而不是直接跳到最终答案；同时把循环状态的数值规模拉回第一轮的水平，防止它越滚越大。结果：一个0.5B的小模型，训练时最多见过32步的题，测试时能解512步的，长度外推16倍，全状态准确率95.5%;而对比方法在同一模型上只有0.8%到55.7%。去掉这两个设计，外推能力立刻缩水到2倍以下。它不是你明天能用上的东西，但它指出了一个方向：想让AI靠“多想一会儿”解决更复杂的问题，得先教会它“每一步想什么”。
📄 原文摘要(英文)
Looped language models can run more iterations at test time, but additional iterations need not help them solve longer problems. For sequential state tracking, final-answer supervision does not specify how far each iteration should advance, and repeated updates can change the scale of the recurrent state. We introduce Progress-Aligned Recurrence (PAR), which trains the prediction at iteration k to match the state after the first k operations and rescales recurrent states to their first-iteration norms. On swap-S_5, a Qwen2.5-0.5B model trained on at most 32 moves achieves 95.5\% mean full-state accuracy at 512 moves, or 16times the training length, compared with 55.7\% reported for RLT and 0.8\% for explicit chain of thought on the same backbone. Controlled S_5 ablations show that removing norm anchoring reduces the extrapolation factor from 16times to below 2.2times, and replacing progress-aligned targets with final-answer supervision reduces it to at most 8times. A learned completion readout selects when to answer without a supplied stopping depth. When intermediate labels are unavailable, prefix self-distillation induces the correspondence from shorter problem prefixes using only final answers. The gains apply across sequential state-tracking tasks, establishing the task structure required for depth extrapolation. These findings support training intermediate predictions and state scale together to make additional recurrent depth useful for longer problems.