大模型能同时想两件事，还能一次生成两段话
大模型内部全是非线性组件，但研究者发现：把两段不同文本的输入按比例混合后，模型输出的下一个词概率，竟然等于分别处理两段文本时输出概率的平均值——像线性叠加一样。这个性质不是训练出来的，而是 Transformer 架构自带的，而且随着预训练推进反而会减弱；但轻量微调就能大幅恢复它。更实用的是，他们设计了一种解码方法，能把叠加的输出拆开，一次前向传播同时生成两段连贯的续写。这意味着未来模型可能并行生成多个候选答案，而不是逐个猜词。
📄 原文摘要(英文)
While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a superposition of the individual next-token distributions. We term this the Superposition Linearity Hypothesis. We provide evidence that superposition is an intrinsic property of the Transformer architecture rather than an emergent consequence of training; in fact, we observe that it tends to diminish as pretraining progresses. However, we demonstrate that linearity can be substantially restored through lightweight fine-tuning, significantly reducing the divergence between the predicted next-token distribution and the average of the individual next-token distributions. Finally, we introduce a guided decoding procedure that disentangles superposed outputs, enabling the simultaneous generation of two coherent continuations from a single forward pass.