大模型带小模型，提升可能是假的
让大模型把「理解」传给小模型、小模型负责快速输出，这套跨模型协作被当成兼顾能力和成本的好办法。研究者搭了个开源框架 PLatour，用零初始化的轻量线性桥把两个冻结模型接起来、直接按任务奖励训练，在生成任务上明显超过开源基线，数学和编程基准上也超过了此前报道的成绩——而且用的是一个更小的发送方。但控制实验拆了台：部分任务的提升，换成无信息的发送信号、自己跟自己通信、或者只训练接收方，都能复现。也就是说，那些可训练的桥可能只是在做参数高效的适配，根本没用到发送方的信息。他们因此提出 PACT-Check，专门量化一套训练好的系统里，有多少提升真的来自「通信」。它不是你明天能用上的东西，但它给所有「大模型帮小模型」的漂亮数字提了个醒：先证明信息真的传过去了。
📄 原文摘要(英文)
Prefill-to-Autoregressive Cross-model Transfer (PACT), a form of latent communication between language models, combines a larger model's prompt representations with a smaller model's efficient generation, offering a promising way to balance capability and inference cost. However, existing systems provide limited support for independently evaluating large-small model collaboration, and improved task performance alone does not establish whether the receiver benefits from the sender's information. We introduce PLatour, an open-source PACT framework that connects frozen models through lightweight, zero-initialized linear bridges trained directly with task rewards and supports hybrid-attention architectures. PLatour significantly outperforms open-source baselines on generation tasks and surpasses Mostik's reported results on shared mathematics and coding benchmarks using a substantially smaller sender. However, controlled experiments show that the gains on some tasks can be reproduced by uninformative sender signals, self-communication, or receiver-only training. We find that trainable bridges can act as parameter-efficient adaptation mechanisms without exploiting useful sender information. We therefore propose PACT-Check to quantify how much of a trained PACT system's gain genuinely comes from communication. Finally, we discuss the opportunities and challenges of PACT systems. Code is available at https://github.com/zhenyi4/platour.