AI Pulse
📄 论文解读

世界模型的新思路:用代码当大脑,视频当眼睛

世界模型一直想模拟环境如何演化,但过去的做法是让 AI 看视频学规律——它看到的是结果,不是规则。这篇论文把两件事拆开了:让一个会写代码的 AI 当“世界大脑”,负责推理事件后果、维护世界状态;再让视频模型当“眼睛”,把状态渲染成画面。中间用一层“代理表示”把时空约束编码成视频,喂给视频模型生成高保真画面。在游戏数据上微调后,模型能按代码定义的规则演化世界,同时保持画面细节。它不是你明天能用上的东西,但方向值得注意:世界模型可能不再靠“看”来理解世界,而是靠“写”来定义世界。

📄 原文摘要(英文)

World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新