AI不是复制粘贴，而是“理解后重写”
我们一直以为大模型做长文本问答时，是从原文里“复制粘贴”答案。但新研究发现，很多关键信息其实是模型“理解后自己写出来的”——它读懂了意思，再用自己的话输出。研究者发明了一种叫LOCOS的方法，能精准定位是哪些注意力头在执行这种“非字面检索”。在Qwen3-8B模型上，只关掉50个这样的头，ROUGE-L分数就从0.401直接掉到0，而传统方法关掉同样数量的头，分数还能保留0.292。更关键的是，这些头只负责检索，不影响模型的算术和记忆能力。这不是你明天能用的工具，但它解释了为什么AI有时会“自由发挥”——它不是在偷懒，而是在用自己的方式理解。
📄 原文摘要(英文)
In long-context use, large language models frequently synthesize answers from the meaning of a relevant context span rather than literally copy-pasting them. Identifying which attention heads perform this synthesis matters for interpreting long-context model behavior. Yet existing detectors miss these heads by construction: they reward heads whose attended token matches the generated token, a literal-copy criterion that captures where a head reads but not what it writes through its output-value (OV) circuit, the very mechanism that carries non-literal retrieval. We introduce Logit-Contribution Scoring (LOCOS), a write-aware detector that scores each head by the projection of its OV-circuit output onto the answer-token unembedding direction, contrasting needle and off-needle source positions in a single forward pass. Across three model families (Qwen3, Gemma-3, OLMo-3.1), mean-ablating the top LOCOS heads on the NoLiMa non-literal retrieval benchmark collapses ROUGE-L at lower head counts than prior attention-based detections; on Qwen3-8B, ablating 50 heads drives ROUGE-L from 0.401 to 0.000 while the strongest baseline still retains 0.292. The selected heads are retrieval-specific: parametric recall and arithmetic reasoning stay at baseline under the same ablation. On Qwen3-8B, the same ablation also drops MuSiQue from 0.55 to 0.08 and BABI-Long from 0.62 to 0.20, while a random-heads control stays within 0.05 of baseline.