AI Pulse
📄 论文解读

AI 造出能玩一分钟的虚拟世界,还记住了你走过的路

现在的 AI 生成视频,大多只能给你几秒钟的片段,镜头一转,场景就忘了。这篇把「生成视频」升级成「生成一个能持续运转的虚拟世界」:你操控角色在开放世界里走一分钟,AI 实时渲染画面,而且它记得你刚才路过的地形、建筑和物体,不会因为镜头转回来就变成另一张脸。做法是把场景的静态几何和动态角色分开建模,再给模型加一个「3D 记忆库」,让它像游戏引擎一样知道哪里有什么。它不是你明天能用上的东西,但这是 AI 从「生成片段」走向「生成可玩的游戏世界」的关键一步。

📄 原文摘要(英文)

Interactive world models extend video generation from offline clip synthesis toward persistent simulation of interactive virtual worlds, enabling applications in games, robotics, embodied agents, and XR. Achieving stable long-horizon interactive generation, however, remains challenging, as the model must simultaneously preserve scene geometry, dynamic consistency, and camera control while supporting real-time autoregressive generation. Building upon Matrix-Game 3.0, we present Matrix-Game 3.5, as shown in Figure 1, which advances real-time interactive world generation toward geometry-aware and long-horizon consistent simulation through three key improvements. First, we propose a unified geometry-aware memory framework, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters, combining explicit 3D patch retrieval with projective camera conditioning to enable geometry-consistent camera control and faithful long-horizon scene recall. Second, we introduce a static-dynamic disentangled world representation that separately models static scene geometry and dynamic subjects, preserving both geometric consistency and subject identity throughout long-horizon generation. Third, we develop a two-stage progressive real-time distillation framework that converts a bidirectional diffusion model into a few-step causal generator through Perceptual Flow Matching and curriculum based Self-Rollout DMD, enabling minute-long real-time interactive generation. Extensive experiments demonstrate that, with a unified training corpus spanning Unreal simulation environments, open-world games, and internet videos, MatrixGame 3.5 achieves strong performance in long-horizon scene recall, precise camera control, subject consistency, prompt-driven world generation, and stable real-time open-world interaction.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新