让AI视频听你指挥:视角、动作、镜头终于分开管了
现在的AI视频生成,你让它“从侧面看角色跑向镜头”,它常常把视角、动作、镜头混成一团,结果画面乱套。这篇把三种控制拆成三个明确的“令牌”,直接放进模型自己的注意力空间里,让它们各自独立又互相协调。效果立竿见影:5B参数的小模型,导航能力追平22B的大模型,切换视角的成功率从18.8%飙到93.8%。它不是你明天就能用的工具,但这是AI视频从“生成画面”走向“可操控世界”的关键一步。
📄 原文摘要(英文)
Interactive video world models require coordinated control over viewing perspectives, character actions, and camera motion. Existing methods typically leave view modes implicit in text captions, conflate action with camera motion, and inject controls through fragmented architectural pathways that capture interdependencies only indirectly. In this work, we present World-in-Context, an interactive world modeling framework powered by In-Context Conditioning (ICC). Rather than relying on specialized conditioning pathways, World-in-Context unifies view, action, and camera controls into explicit tokens contextualized directly within a shared self-attention space. Our formulation reveals that (i) explicit view tokens disentangle conflicting dynamics and unlock emergent view switching from single-view data, while (ii) action and camera inputs play complementary roles in governing local responsiveness and global stability. To preserve temporal continuity across execution intervals, we further introduce Interaction-Transport RoPE (IT-RoPE), parameterizing temporal attention with accumulated control transitions. On WBench, World-in-Context-5B achieves navigation performance on par with 22B EchoWM (81.5 vs. 81.6) and attains a 93.8% perspective-switching success rate, substantially outperforming existing 5B baselines (18.8%). Controlled analyses further demonstrate compounding advantages of World-in-Context under richer conditioning mixtures, favorable data efficiency, and natural applicability to causal rollouts.