游戏AI终于能记住自己刚才干了啥
现在的游戏AI生成画面，是走一步算一步，像金鱼一样记不住自己刚才的位置和姿势，所以跑久了就会穿模、两个人越走越远。这篇把AI的脑子拆成两半：一半专门记“世界状态”(谁在哪、什么姿势、朝哪走)，另一半只负责把状态画成好看的画面。结果就是，AI终于知道自己刚才在哪了，而且你可以直接改它的状态——比如加一条“不许穿地”的规则，穿模直接减少66%。这不是你明天能玩到的游戏，但它解决了AI生成游戏画面最要命的“记性差”问题。
📄 原文摘要(英文)
Interactive game world models typically autoregress visual observations directly in pixel or latent space, forcing structured properties such as pose, geometry, and occlusion to be implicitly maintained by the same generative sequence. Over long horizons, errors in these latent world properties accumulate, making consistency and controllability fragile. We explicitly model the evolving world state, delegate exact geometric computation to a fixed, zero-parameter renderer, and leave the neural model to synthesize appearance. We instantiate this idea as Marionette, a world model for interactive games with articulated characters. First, a two-stage autoregressive dynamics model predicts an explicit and interpretable 276-dimensional 3D world state comprising multi-entity articulated skeletons, metric root trajectories, and rotations. Second, a zero-parameter graphics bridge converts the predicted state into pose-control videos, computing world-space geometry and occlusion in closed form. Third, a control-conditioned video-diffusion observation model synthesizes photorealistic RGB observations from the resulting structured controls. Our experiments establish two properties of Marionette. First, the predicted world state is directly controllable. Forcing a mismatched action stream changes root-aligned joint error by 31% across 48 held-out segments. Second, long-horizon behaviour is determined in the state, and can be repaired there. Left free, the two generated characters drift to 21.2 m apart (recorded sessions stay near 5 m) and a third of frames show ground penetration. Two rules imposed on the explicit state, a terrain collider and a separation cap, cut penetration by 66% and keep the pair engaged, with no change to the observation model. Routing appearance through the predicted state costs no fidelity we can detect, at an FVD of 831 against 799 for recorded pose.