AI Pulse
📄 论文解读

AI生成视频开始先搭3D骨架

让AI生成视频,现在多半是「一句话进去、一段画面出来」,你没法指定谁站在哪、往哪动。这篇换了个路子:先让AI自己写代码搭一个粗糙的3D场景当「骨架」,再让视频模型照着这个骨架去渲染,结构和运动因此变得可控。他们做了对照实验加盲评人评,发现骨架的粗细要分任务取舍:粗骨架运动更自然,中等粗细更保真;再加上物理模拟、让提示词对齐骨架,平均质量还能往上走。落到产品上是一个会看任务选路子的框架:该搭骨架就搭,该直接生成就直接生成,每个物体用多细的建模各自决定,还配了一组AI分工——保存场景素材、靠看画面来修错、互相独立复核。评测里它在指令遵循和场景一致性上领先,物理合理性比直接生成好,真实感强过同类骨架方案;长视频里跨多次交互的事件结构和一致性也有提升,但镜头切换的稳定性和对场景设定的遵守仍是短板。做视频工具、想用AI批量出分镜或广告片的人,这是「可控生成」这条路上一个值得盯的进展,还不是你明天打开就能用的成品。

📄 原文摘要(英文)

Code2Video uses agent-authored 3D proxies to guide video generation, enabling explicit control over structure and dynamics. However, how to construct proxies and prompts for optimal downstream quality remains underexplored, as do the capabilities and limitations of this paradigm. Through controlled experiments and blinded human evaluation, we study proxy granularity, physics simulation, and prompting. The results reveal task-dependent trade-offs: coarse proxies favor natural motion, while medium-grained proxies favor consistency. Physics simulation and proxy-aligned prompting further improve average quality. Guided by these findings, we introduce Coda-DaVinci, a task-adaptive agentic framework for Code2Video generation that selects between proxy-guided and direct generation, adapts modeling granularity to each entity, and coordinates simulation with proxy-grounded prompting. Its multi-agent harness preserves scene assets and guides repairs through visual inspection and independent review. Experiments demonstrate leading instruction following on VTR and scene consistency on R2M, improved physical plausibility over direct generation, and higher visual realism than the compared Code2Video baselines. Long-video evaluations further show gains in event structure and consistency across successive interactions, while revealing limitations in transition stability and adherence to scene settings.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新