AI 记忆太满会卡壳，这篇让长对话提速 10 倍
AI 助手聊得越久，记的东西越多，算得就越慢——这是长对话卡顿的根源。SparseEngine 换了个思路：不再把所有历史都原样存着，而是让 AI 自己决定哪些该留、哪些该丢，同时保证丢完还能接得上话。它把 15 种省内存的方法统一进一个引擎，实测吞吐量比主流方案 vLLM 高 10 倍，解码快 2.5 倍，跑智能体任务整体快 2 倍。这不是你明天就能装上的东西，但它指向一个趋势：AI 的“记忆力”正在从“全记住”转向“会取舍”，长对话的流畅度会因此上一个台阶。
📄 原文摘要(英文)
Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache representations and workflows hinder integration with existing inference engines, while prior sparse-serving abstractions support only specific layouts or workflows. We present SparseEngine, a ground-up, sparse-first inference engine whose shared lifecycle contract lets each method control its KV representation and computation while coordinating state transitions with common serving infrastructure. SparseEngine supports 15 methods across four categories and enables cross-request state management through Chain Cache, which resumes KV-eviction methods from retained history, and controllable Prefix-Cache Pruning, which removes KV from selected history regions while preserving logical-prefix matching. While maintaining method quality, SparseEngine delivers over 10x higher throughput with KV eviction, over 2.5x faster decoding at matched concurrency than vLLM, and over 2x end-to-end speedup on agent benchmarks. The code is available at https://github.com/CURRENTF/SparseEngine.