AI 的“等一下”不是学来的，是继承的
大模型在推理时写出的“wait”(等一下)常被归功于强化学习教出来的。这篇论文把“wait”这个词在开源模型训练史里从头查了一遍：没经过强化学习的模型几乎从不自己发明这个词；但训练数据里一旦有，强化学习会把它的使用量放大好几倍。更关键的是，强化学习本身带来的准确率提升，完全不依赖这个词——模型没有它照样变聪明。研究者还发现，用 AI 当裁判去评估推理质量，可能把人工标注的趋势都反过来。它不是你明天能用上的东西，但它提醒你：我们看到的 AI“思考痕迹”，很多是训练数据里的习惯，不是模型自己长出来的能力。
📄 原文摘要(英文)
Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models' training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3's recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.