KV缓存压缩的评分机制几乎没用，随机淘汰就行
大模型做长推理时，中间思考过程会撑爆内存，现有压缩方法都在给每个缓存token打分、留高分。这篇论文发现：打分本身几乎没用。随机淘汰——只要保住提示词部分，推理过程里的token随便丢——效果和最强选择器持平，吞吐量还高32-43%。原因是推理痕迹自带冗余：模型会反复重述它还需要的东西，而且每个注意力头各留一份副本，随机抽也够用。它不是你明天能用上的，但它在说：我们可能高估了KV缓存压缩里“聪明选择”的价值，真正的脆弱点只有提示词。
📄 原文摘要(英文)
Large language models achieve superior performance on tasks that require extended reasoning, but long chains of thought make the KV cache a severe memory bottleneck. Existing KV cache compression methods share one paradigm: score each cached token by some estimate of how much it will matter later, and keep the top-scoring ones. We show that the selection signal contributes almost nothing. Random Attention keeps the prompt and evicts uniformly at random within each attention head, computing no score at all; across four models and six reasoning tasks it matches the strongest prior evictor while serving 32-43% higher throughput than it in vLLM deployment. Controlled experiments explain this by showing that 1) the prompt is the fragile part of the cache, and most of the gap between selectors is just whether their selection signal happened to keep it; 2) the reasoning trace protects itself against eviction with redundancy at two levels, in the text (the model restates what it still needs as it works) and across attention heads (each keeps its own copy of the trace), so once the prompt is safe, a random draw retains enough copies of what the model still needs, and no score is required to pick them. Our code is publicly available at https://github.com/SalesforceAIResearch/Random-Attention.