让AI先写代码再拍视频,物理规律终于不崩了
现在的AI视频生成,画面漂亮但物理常崩:杯子掉下去会穿桌,人走路像飘。原因在于模型只能从一句文字提示里猜整个时间过程。这篇的思路是:先让一个AI写出一段Blender 3D软件的代码,把场景和物体怎么动、怎么碰都明确写出来,再用另一个AI把这段代码渲染出的粗糙画面变成逼真视频。相当于把「想清楚物理」和「画得好看」拆给两个引擎各干各的。在物理一致性基准上,分数从0.475涨到0.558,在另一个综合基准上从52.18涨到77.88。它不是你明天就能用的工具,但指向一个更靠谱的方向:让AI用可检查的代码当草稿,而不是直接瞎猜。
📄 原文摘要(英文)
Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.