让AI先写代码再拍视频,物理规律终于不崩了
现在的AI视频生成,你输入“苹果从桌上滚落”,它靠猜的,经常滚着滚着就穿模或飘起来。这篇换了个思路:先让AI用Blender(3D建模软件)写一段代码,把场景和物体运动轨迹精确算出来,再让视频模型照着这个“草稿”渲染成逼真画面。相当于先画好分镜和物理计算,再拍电影。在物理一致性评测上,分数从0.475涨到0.558,在另一个综合评测上从52.18涨到77.88。它不是你明天就能用的工具,但指明了方向:让AI用可执行的代码当“思考过程”,比让它直接猜更靠谱。
📄 原文摘要(英文)
Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.