AI世界模型终于学会“眼观六路”
现在的AI世界模型都“近视”——只盯着主角眼前那点画面，东西一离开视线就凭空消失，再回来时状态全错。这篇让AI学会“眼观六路”:除了主角视角，再生成一个或多个全景观察者，专门盯着你离开的地方。物体在你背后继续演化，等你回头时它已经按物理规律变好了。做法是把主角和观察者都从同一个全景画面里“裁”出来，保证几何位置对得上；物体重新入画时，再用高分辨率参考把细节补回来。观察者可以随便放、放多个、还能用控制信号指挥它看哪。这不是你明天能用的功能，但它是世界模型从“游戏视角”走向“真实世界”的关键一步——真实世界不会因为你没看就暂停。
📄 原文摘要(英文)
How can a world model continuously observe regions beyond the actor's current view? Video world models simulate how an environment evolves from an agent's actions, yet remain actor-centric. Once an object leaves the actor's view, they lose direct evidence of its evolution, often failing to preserve its state and dynamics upon re-entry. To address this, we introduce World Observer, which decouples observing from acting by jointly generating a perspective actor for the agent-centric view with one or more panoramic observers that watch selected world regions. This allows objects that leave the actor's view to remain visually evolving in an observer, so their updated states are reflected when they re-enter. We ground the actor and observers by warping from a shared panoramic source for explicit geometric correspondence, and introduce an Observer Sink of high-resolution perspective references to restore fine appearance upon re-entry. Since the observers are decoupled from the actor, they can be placed freely across the scene, extended to multiple locations for broader coverage, and driven by control signals to steer out-of-view evolution. To evaluate out-of-view evolution, we further introduce world-space metrics and a benchmark spanning real and synthetic scenes. World Observer substantially improves out-of-view dynamics while remaining competitive in visual fidelity, camera control, and 3D adherence.