AI Pulse
📄 论文解读

AI 造出能玩一分钟的虚拟世界,还不会穿帮

现在的 AI 生成视频,大多只能给你几秒钟的片段,镜头一转、物体一动就崩。这篇把「生成视频」升级成「生成一个能持续运转的虚拟世界」:模型内部多了一块「几何记忆」,记住场景里每个东西的位置和长相,你转动镜头时,它知道该看到什么,而不是重新瞎编;同时把「静止的背景」和「会动的角色」分开建模,背景不会因为角色走动而扭曲,角色也不会因为镜头移动而变脸。配合一套把生成速度压到实时的蒸馏技术,它能在开放世界里连续生成一分钟以上、画面稳定、镜头听你指挥的交互视频。它还不是你明天能玩上的产品,但这是「AI 生成可交互游戏世界」从演示走向引擎的关键一步。

📄 原文摘要(英文)

Interactive world models extend video generation from offline clip synthesis toward persistent simulation of interactive virtual worlds, enabling applications in games, robotics, embodied agents, and XR. Achieving stable long-horizon interactive generation, however, remains challenging, as the model must simultaneously preserve scene geometry, dynamic consistency, and camera control while supporting real-time autoregressive generation. Building upon Matrix-Game 3.0, we present Matrix-Game 3.5, as shown in Figure 1, which advances real-time interactive world generation toward geometry-aware and long-horizon consistent simulation through three key improvements. First, we propose a unified geometry-aware memory framework, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters, combining explicit 3D patch retrieval with projective camera conditioning to enable geometry-consistent camera control and faithful long-horizon scene recall. Second, we introduce a static-dynamic disentangled world representation that separately models static scene geometry and dynamic subjects, preserving both geometric consistency and subject identity throughout long-horizon generation. Third, we develop a two-stage progressive real-time distillation framework that converts a bidirectional diffusion model into a few-step causal generator through Perceptual Flow Matching and curriculum based Self-Rollout DMD, enabling minute-long real-time interactive generation. Extensive experiments demonstrate that, with a unified training corpus spanning Unreal simulation environments, open-world games, and internet videos, MatrixGame 3.5 achieves strong performance in long-horizon scene recall, precise camera control, subject consistency, prompt-driven world generation, and stable real-time open-world interaction.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新