AI Pulse
📄 论文解读

AI 造世界终于能边玩边生成,不再卡顿

你玩过那种 AI 生成的游戏世界吗?以前它要么记不住你走过的路,要么为了记住路把电脑卡死。这篇论文让 AI 造世界第一次做到了实时:它把场景压缩成一种叫「3D 高斯」的小块,只在你探索新地方时才新增记忆,而不是每走一步都存一帧。结果就是,生成速度比之前最快的方案快 359 倍,能到每秒 9 帧,边玩边生成不卡壳。它不是你明天能玩上的游戏,但这是「AI 无限世界」从演示走向真游戏的关键一步。

📄 原文摘要(英文)

Interactive world models generate visual content conditioned on user actions, yet maintaining spatial consistency over long-horizon exploration in real time remains challenging. Existing approaches rely on either temporal context or explicit 3D memory, both of which scale poorly because memory grows with the length of the generated stream. Moreover, explicit 3D memories condition latent video generators by re-encoding RGB renderings, which precludes real-time generation. In this paper, we introduce a Latent 3D Gaussian Memory (L3G) that combines the efficiency of feed-forward reconstruction with a compact, rapidly renderable scene representation. Building on this memory, we propose L3G-World, in which a memory updater and a video diffusion transformer (DiT) alternate autoregressively to generate spatially consistent worlds. The memory updater spawns Gaussians in newly observed regions and refines existing ones, so that the memory grows with scene extent rather than stream length. Each Gaussian further stores a VAE latent feature, allowing the memory to be rendered directly into the DiT's latent space without VAE re-encoding. Experiments show that L3G-World achieves state-of-the-art generation quality and long-horizon spatial consistency, with up to 38.6times fewer 3D primitives, 3.2times faster memory updates, and 359times faster memory rendering relative to the most efficient prior 3D-memory baseline for each metric, enabling real-time world generation at 9 FPS.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新