让AI看懂电影：把视频拆成知识图谱再拼回去
现在的AI生成视频，大多只是“看”像素，不懂镜头语言。这篇论文让AI把视频拆成一张知识图谱：谁在做什么、镜头怎么切、声音和画面怎么配，都写成文字和素材节点，再用这些节点把视频重新拼出来。拼得越像，说明它理解得越深。结果，用这套方法训练的AI，在生成电影级视频的任务上比最强基线高了20.7个百分点，而且用的提示词token少了74.3%。它不是你明天就能用的工具，但这是AI从“会画”走向“会拍”的一个关键方向。
📄 原文摘要(英文)
Creative agents still lack an effective way to learn from high-quality human films, limiting their ability to produce cinematic-grade videos. A key challenge is the absence of a structured video representation that is both faithful to film content and directly usable for agentic reasoning and manipulation. To address the challenge, we propose the Agentic Video Auto-Encoder (AVA-Encoder), a framework for learning agent-native video representations via agentic auto-encoding. AVA-Encoder transforms a video into a knowledge graph (KG) representation and then reconstructs it back into video. Its hierarchy and state nodes store structured text, while a linked asset layer holds generated images, audio, and video. Typed edges preserve the relations between these text descriptions and assets in a form that agents can easily understand, query, and edit. The video reconstruction differences drive a textual-gradient optimization framework, which expresses evaluation feedback as natural-language update directions for Data-Independent Encoding Policy Pseudo-Training in the outer loop and optional Data-Dependent KG Representation Refinement in the test-time inner loop. Extensive experiments show that AVA-Encoder improves by 20.7 percentage points over the strongest external baseline. In the controlled policy-only setting, its pseudo-trained shot-level Agentic Video Encoder policy also outperforms a carefully human-tuned policy while using 74.3% fewer system-prompt tokens. We release the complete AVA-Encoder framework, a reliable agentic video reconstruction benchmark, and the first dataset of high-quality film KG representations.