视频世界模型终于能听你指挥了
现在的视频生成模型能凭空造出连续画面，但你想让它「换个角度看同一个场景」，它基本不听你的——要么画面崩了，要么视角没变。这篇提出一个不用重新训练、直接套在现有模型上的接口：它把「你想从哪个角度看、看到哪一块」拆成几路信号，分别喂给模型，再用几何信息保证这些信号不打架。结果是你真能像拖拽镜头一样，让模型从新视角补全画面、回头看之前的地方也不穿帮，甚至能把一个人的动作迁移到另一个场景里。它不是你明天就能用的产品，但这是视频生成从「会画」走向「会拍」的关键一步。
📄 原文摘要(英文)
Autoregressive video world models enable interactive, long-horizon exploration, but flexible control remains challenging. Exploring a source video from new viewpoints requires the generated rollout to remain synchronised with the recorded event, place observed content in the requested view, plausibly complete newly exposed regions, and recover previously generated appearance on revisits. Existing methods typically address these requirements through task-specific modules or additional training. We present World in World, a training-free inference-time interface that converts heterogeneous control evidence into camera- and time-labelled clean visual states, which are read through the native self attention of a frozen causal video model. The evidence comprises source-video observations, target-view scene projections, geometry renderings that guide completion of newly exposed subject regions, and retrieved generated states beyond the rolling cache. Each evidence source carries token-level support and its own availability schedule. A correspondence router combines persistent point identities with geometry to establish token correspondences, guiding supported queries towards matching source-video tokens. Evidence-wise attention CFG (EWA) then independently regulates each auxiliary channel's additional contribution using attention responses from the same denoising forward pass. The shared interface supports camera-controlled rerendering, long-horizon revisiting, and human-motion transfer with the same frozen backbone. We evaluate World in World on camera-controlled video rerendering under diverse viewpoint changes, assessing perceptual quality, temporal consistency, and camera-following accuracy.