视频生成终于能记住主角长什么样了
现在的 AI 生成视频，主角出场三秒后换个角度，脸就变了——因为它只记得最近几帧，早把开头忘光了。这篇论文给视频模型装了个「记忆路由器」:不是把历史画面一股脑塞回去，而是先判断当前画面需要哪段旧记忆，再只把这段记忆注入模型里专门负责「认东西」的几层，其余部分照常处理。结果是在 100 个多镜头测试里，长视频的角色、场景、物品能跨镜头保持一致，而且推理速度几乎没变。它不是你明天就能用的工具，但这是视频生成从「每帧都好看」走向「整段都连贯」的关键一步。
📄 原文摘要(英文)
Autoregressive video diffusion enables scalable long-video generation by producing chunks from a bounded recent context. While recency-based caching preserves local continuity, it evicts historical cues needed when subjects, objects, scenes, or attributes reappear. Existing memory mechanisms expose models to nonlocal history, but access alone does not ensure effective use. Our analysis reveals that video DiT layers exhibit distinct preferences for current, recent, and distant context, suggesting that long-range memory requires deciding both what to retrieve and where to use it. We introduce LayerRecall, a current-conditioned, layer-selective memory router that retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers while preserving local attention elsewhere. To reduce reliance on scarce high-quality long-horizon videos and explicit memory-allocation labels, we further propose Cross-Horizon Prediction Matching (CHPM), which uses a privileged long-context reference to supervise the bounded-memory router in prediction space. Across 100 multi-shot evaluation prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity. Qualitative analyses further reveal memory-guided self-correction, whereby initially mismatched local attributes return to their historical appearance without resetting ongoing motion or scene structure. Additional analyses show cross-backbone portability and negligible inference overhead.