AI生成视频正在失忆,这篇论文想给它装上记忆
现在的AI生成视频有个隐蔽的硬伤:它记不住自己生成过什么。你让它生成一个长视频,角色走到第50帧,可能已经忘了自己第10帧长什么样,于是脸变了、衣服换了、场景逻辑崩了。原因很朴素——模型的计算窗口是有限的,早期的信息早就被挤出去了。这篇综述把这个问题正式命名为「记忆问题」,并系统梳理了学界正在怎么解决:有人把历史信息压缩成向量存起来,有人专门设计读写机制,有人让模型在生成过程中自己学会该记住什么。它不是你明天能用上的功能,但它划出了一条清晰的赛道:谁先解决「长期记忆」,谁才能做出真正连贯的长视频世界。
📄 原文摘要(英文)
Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environments. Autoregressive (AR) video generation extends visual sequences through causal rollouts. However, a fundamental bottleneck emerges: as the generated sequence expands, practical models must operate under strictly bounded context windows, storage, and computational limits. Consequently, critical historical information, e.g., entity identities, dynamic states, and intervention-induced causal changes, often leaves the active context long before its relevance diminishes. Overcoming this limitation and maintaining temporal persistence constitutes a fundamental memory problem. We present a systematic and comprehensive review of memory mechanisms in AR video generation. We formulate memory operationally as persistent historical information maintained across outer AR steps, capable of influencing future generation even after the originating evidence is no longer locally accessible. Building upon this unified framework, we organize the literature through five complementary perspectives: (I) Forms, the representational carriers of history; (II) Functions, the specific semantic and physical information requiring preservation; (III) Operations, the lifecycle of writing, reading, updating, managing, and integrating memory; (IV) Learning, the optimization of memory behaviors under closed-loop rollouts; and (V) Evaluation, the paradigms for diagnosing genuine memory capabilities. We conclude by synthesizing open challenges, including composable and resource-aware memory architectures, trustworthy state updating, self-rollout learning, and standardized evaluation. By bridging representations, mechanisms, and learning paradigms, this paper establishes a structured foundation for developing reliable, memory-conditioned video generation systems.