AI Pulse
📄 论文解读

AI 把世界存成一张可改的全景图

现在的视频世界模型把「世界」塞在生成器里,你走开再回来,它得重新画一遍,之前改过的东西可能就漂了、没了。这篇把世界状态挪到生成器外面:存成一张全景图,外加一份修改记录,由多模态大模型来管——它把每次请求翻译成对这张图的操作,自己挑局部改动写在哪,每个候选先检查再落成下一版。生成器只负责提视图,视频只负责展示。于是你走到新地方再走回来,同一版本下看到的和之前一模一样;改一个物体,区域外的像素一个不动。评测里它在四个环境还原指标上全部最高(极线一致性 91.5,对手 0.1 到 74.0),相机往返完成率 85.3%,对手最多 50.9%;112 次盲评局部编辑,先画完再选写入区域把完成率从 34.8% 拉到 83.0%。做虚拟场景搭建、想在一个地方反复改布局又不希望别处跟着变的人,这是能直接用的那种。

📄 原文摘要(英文)

Video world models keep the world in the context of a generator, so a revisit is synthesized again and earlier edits can drift or disappear. We present WM^MLLM, the first interactive world model built on multimodal large language models (MLLMs), for persistent, editable world generation from a single image. WM^MLLM keeps the world state outside every generator, as a panorama with a revision history, and lets the MLLMs write it: they turn each request into operations on this state, select where a local edit is written, and check every candidate before it becomes the next state. Image generators only propose views, and video only presents them. Users can walk to new places and return, edit objects through text or a box, place subjects, and change lighting, weather or season. By construction, a revisit at the same revision reproduces the earlier view exactly, a local edit leaves every panorama pixel outside its region unchanged, and each request starts from the result of the previous one. Against six world model baselines on MBench-A, WM^MLLM has the highest mean on all four environment-recovery metrics (Epipolar 91.5 versus 0.1 to 74.0) and completes the most camera round trips (85.3% versus at most 50.9%). It has the best value in all 15 CompBench columns against 15 baselines. On 112 local edits judged blind, selecting the region to write after generation raises completion from 34.8% to 83.0%. On 258 WBench and MBench-A items, all 46 revisits recover their earlier views, and all 97 local edits that pass the check leave panorama pixels outside their region unchanged.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新