AI推理时，内存不够？新方法只留关键记忆
大模型在回答长问题时，需要记住之前的所有内容，这就像一个人边读边记，但记忆空间有限。现有方法要么凭经验丢弃旧信息，要么用粗略的分数判断，结果常常丢错关键点。KVpop 的做法是：让模型自己学习“哪些记忆未来还会用到”，并专门训练一个评分器，在丢弃前先“延迟几步”观察，确保留下的都是真正重要的。在数学推理测试中，它只保留 25% 的记忆，就能达到原来 98% 的准确率。这不是你明天能直接用的工具，但它指向一个趋势：未来的 AI 会更聪明地管理自己的“工作记忆”，而不是一味扩大内存。
📄 原文摘要(英文)
Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on static heuristics or proxy scores, which poorly track future token utility and cause brittle eviction as relevance shifts. To address this, we introduce KVpop, which learns a fixed-budget KV eviction policy by directly supervising the keep-or-drop decision. The scorer is trained against a novel future-attention target, computed efficiently without materializing dense attention maps. We further introduce a delayed memory-based scorer that, uniquely among learned eviction methods, defers scoring for a fixed number of steps to exploit near-future context. On AIME and HMMT mathematical reasoning, KVpop retains 98% of full-attention performance on Qwen3-4B at 75% KV cache compression and 97% at 88% compression, consistently outperforming established eviction baselines. Qwen3-8B shows even stronger results, reaching near-full teacher performance. These results show that supervising eviction with future-attention signals cuts memory costs while maintaining quality.