视频生成正在失忆,这篇论文想给它装上记忆
现在的 AI 生成视频,画面越长越容易「忘事」:主角的脸、物体的位置、你中途改过的设定,生成到后面就丢了。这不是画质问题,是记忆问题——模型只能看到眼前一小段,早先的关键信息早被挤出了窗口。这篇综述把「记忆」正式定义成一种能力:即使原始证据已经不在眼前,历史信息仍能影响后续生成。它把散落的做法整理成五个维度:记忆长什么样、要记什么、怎么写入和读取、怎么在自我滚动中学习、以及怎么评测它到底有没有真记忆。它不是你明天能用上的工具,但它把「视频生成为什么越做越笨」这件事,第一次讲成了可以系统解决的工程问题。
📄 原文摘要(英文)
Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environments. Autoregressive (AR) video generation extends visual sequences through causal rollouts. However, a fundamental bottleneck emerges: as the generated sequence expands, practical models must operate under strictly bounded context windows, storage, and computational limits. Consequently, critical historical information, e.g., entity identities, dynamic states, and intervention-induced causal changes, often leaves the active context long before its relevance diminishes. Overcoming this limitation and maintaining temporal persistence constitutes a fundamental memory problem. We present a systematic and comprehensive review of memory mechanisms in AR video generation. We formulate memory operationally as persistent historical information maintained across outer AR steps, capable of influencing future generation even after the originating evidence is no longer locally accessible. Building upon this unified framework, we organize the literature through five complementary perspectives: (I) Forms, the representational carriers of history; (II) Functions, the specific semantic and physical information requiring preservation; (III) Operations, the lifecycle of writing, reading, updating, managing, and integrating memory; (IV) Learning, the optimization of memory behaviors under closed-loop rollouts; and (V) Evaluation, the paradigms for diagnosing genuine memory capabilities. We conclude by synthesizing open challenges, including composable and resource-aware memory architectures, trustworthy state updating, self-rollout learning, and standardized evaluation. By bridging representations, mechanisms, and learning paradigms, this paper establishes a structured foundation for developing reliable, memory-conditioned video generation systems.