AI读长文不再爆显存：只留关键记忆
大模型处理超长文本（比如整本书）时，GPU内存会被历史数据撑爆。这篇论文反直觉地提出：与其被动记住所有内容，不如主动预测未来需要什么，只保留关键片段。他们设计了一个独立的“记忆索引器”，像图书管理员一样提前判断哪些信息会被用到，然后只加载这些。在500K长度的极端测试中，内存占用减少90%以上，准确率反而微升。这不是你明天能用的技术，但它指向一个趋势：未来的AI会更像人脑，懂得遗忘才能高效思考。
📄 原文摘要(英文)
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.