AI Pulse
📄 论文解读

AI终于有了真正的记忆,不再每次重头开始

现在的AI每次对话都像失忆症患者:你昨天跟它聊了什么,它今天全忘了,只能靠把整段历史重新塞给它来“回忆”,又慢又费钱。这篇论文的团队做了一个叫MemT-1.0的模型,把“记忆”直接焊死在模型的计算过程里——它每处理一句话,就自动更新一个内部的记忆状态,像人脑一样持续累积,而不是每次从头读一遍。在128K长度的对话历史下,它比传统方式快1.86倍,存储占用更是只有原来的1/66。更关键的是,它能把之前学到的“操作经验”内化成能力,在让AI执行多步任务的场景里成功率明显提升。这不是你明天就能在ChatGPT里用上的功能,但它指向一个明确的方向:AI正在从“每次重来”走向“越用越懂你”。

📄 原文摘要(英文)

Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86times end-to-end inference speedup and a 66times reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新