代码造世界,视频模型只负责渲染?
视频生成模型一直被当成「世界模拟器」,这篇却把它降级成「渲染器」:让代码先搭好 4D 世界(物体怎么动、镜头怎么走),视频模型只负责把画面画出来。研究者做了 432 个严格配对的测试案例,横跨 48 个可执行的 4D 世界,在 12 个主流模型上测了一轮,结果不太好看。第一,保住世界结构和画对样子这两件事在操作上互相拉扯,很难同时做好;第二,给的草稿越丰富,结构越稳,画面保真度反而越差;第三,源素材线索一冲突,模型就严重「抄错源」,泄漏率最高到 64%,说明它跨模态拆解信息的能力还很脆。他们顺手提了个即插即用的视觉提示方法,在冻结的渲染器上把两项指标都拉高了一点。它不是你明天能用上的东西,但它指了个方向:以后做 4D 预演、动画分镜、虚拟拍摄的人,可能不用再让一个模型既当导演又当画师。
📄 原文摘要(英文)
Multimodal coding agents decouple 4D world construction from visual generation, repurposing video models from world generators toward generative renderers. This division of labor raises a fundamental question: if code builds the world, can video models just render it? A reliable generative renderer hinges on two complementary criteria: (i) world preservation, retaining prescribed 4D kinematics and camera paths; and (ii) visual realization, persistently binding text-specified target attributes over time. To evaluate this capability, we introduce WorldDraft, a benchmark of 432 strictly paired test cases across 48 executable 4D world families spanning six dynamic challenge categories. Under a 3 times 3 factorial design, WorldDraft orthogonally crosses draft granularity (abstract proxy to previs) and appearance alignment (neutral to conflicting) under identical dynamics. Extensive evaluations across 12 representative models reveal that: (I) world preservation and visual realization remain in stark operational tension; (II) richer draft scaffolding anchors structure but paradoxically impairs appearance fidelity; and (III) conflicting source cues trigger severe source leakage (up to 64%), exposing fragile cross-modal disentanglement. To mitigate these bottlenecks, we propose WorldDraft-Primer, a plug-and-play visual prompting strategy that consistently improves both objectives across frozen video renderers. Together, WorldDraft and WorldDraft-Primer lay a foundation for bridging programmatic 4D simulation with video generation, advancing physically grounded generative rendering.