AI Pulse
📄 论文解读

让AI先写代码再拍视频,物理规律终于不崩了

现在的AI视频生成,画面漂亮但物理常崩:杯子掉下去会穿桌,人走路像飘。原因在于模型只能从一句文字提示里猜整个时间过程。这篇的思路是:先让一个AI写出一段Blender 3D软件的代码,把场景和物体怎么动、怎么碰都明确写出来,再用另一个AI把这段代码渲染出的粗糙画面变成逼真视频。相当于把「想清楚物理」和「画得好看」拆给两个引擎各干各的。在物理一致性基准上,分数从0.475涨到0.558,在另一个综合基准上从52.18涨到77.88。它不是你明天就能用的工具,但指向一个更靠谱的方向:让AI用可检查的代码当草稿,而不是直接瞎猜。

📄 原文摘要(英文)

Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新