AI Pulse
📄 论文解读

AI视频模型记不住东西?问题不在画质在记忆

AI生成视频时,镜头离开再回来,场景里的物体可能悄悄变了——这不是画质问题,是记忆问题。研究者设计了一套标准测试,把记忆机制单独拎出来比:原始上下文(把历史帧全存着)在开放场景中表现最好,而压缩记忆会丢失关键证据。最意外的是,用状态空间模型做隐式记忆,效果甚至超过显式存帧。结论:记忆的结构比容量本身更重要。这不是你明天能用的技术,但它解释了为什么AI视频总在“穿帮”——不是画不好,是记不住。

📄 原文摘要(英文)

We present Echo-Memory, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text prompt, and camera-action sequence, but their central failure is often memory rather than local image synthesis: after the camera leaves and returns, the scene or salient object may silently change. Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences. Echo-Memory fixes the action-to-video interface and varies only how history is stored and read by the generator. Under a shared video diffusion backbone, optimizer, camera-action representation, sampler, and evaluation pipeline, we compare raw context, compression-based memory, spatial summaries with different read-out paths, and state-space recurrence. This matched matrix separates four otherwise conflated axes: capacity, compression, read-out, and recurrence. We also evaluate memory through a three-branch protocol: replay quality, in-domain loop revisit, and open-domain return probes. The branches routinely disagree, showing that replay fidelity is not a sufficient proxy for remembering a world. Three findings follow. Raw context is a strong capacity baseline and improves open-domain return far more than it improves replay metrics. Compactness is not a free substitute for capacity: aggressive spatial and hybrid-compression memories lose the salient evidence needed for return. Finally, block-wise state-space recurrence is the strongest open-domain return mechanism in our matrix, showing that the structure of implicit memory matters as much as the decision to use it. These results provide a compact protocol for studying memory in action world models beyond isolated replay metrics.

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部