大模型想太久会爆内存，这篇用「路标」提前预判
大模型做长推理时，每一步都要把之前想过的内容存下来，想得越久，内存爆得越快。现有压缩办法靠「最近在想什么」来猜「接下来会看什么」，但长推理里有个反直觉的现象：模型会突然回头重看很早之前定下的解题计划，这种「回头」用最近的思路根本猜不到。研究者发现，这些回头看的时刻在模型内部其实聚成了少数几类，于是他们给每类放一个「路标」查询，不用记全部历史，只靠路标就能预判哪些旧内容会被重新翻出来。结果内存最高省到原来的六分之一，速度提升四倍多，准确率几乎不掉。它不是你明天能装进自己项目里的东西，但它指出了一个方向：与其让模型无限回忆，不如教它学会「什么时候该回头」。
📄 原文摘要(英文)
Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) cache grows linearly with sequence length and creates severe memory bottlenecks, often exceeding GPU capacity for long reasoning traces. Existing KV cache compression methods rely on recent queries to estimate future token importance, implicitly assuming these serve as reliable proxies for future attention patterns. We demonstrate that this assumption fails in long-horizon reasoning: certain decoding steps generate Thought Revisiting Tokens (TRT) that re-attend to distant previous context, such as task-solving plans formulated early in the trace. Through systematic analysis, we discover that queries corresponding to the TRT cluster into a small number of similarity groups in the embedding space. Based on this insight, we propose BeaconKV, a training-free KV cache compression method that maintains beacon queries, compact representatives for each global query cluster, to anticipate which KV pairs will be revisited without storing the entire query history. Across four open-source LRMs and diverse reasoning benchmarks, BeaconKV generally outperforms existing compression methods, achieving up to 5.8times memory reduction while nearly preserving full cache accuracy and improving throughput by over 4.3times.