AI 写代码拍视频,比直接生成更靠谱
现在的 AI 视频模型能生成惊艳片段,但让它精确渲染一个公式、一张正确的图表、一个规则一致的游戏,它就崩。这篇换了一条路:让 AI 写代码,一帧一帧渲染、一个音一个音合成。研究者做了个基准测试,239 个任务,从数据故事到课堂讲解,要求 AI 在一个没有图像/视频生成模型的沙盒里,一小时内交付带声音的成品 MP4。结果:最强的系统在 76.3% 的任务上满足所有核心要求——这个成功率远超直接生成式模型。但有个反直觉的发现:AI 会自己搭渲染管线、检查自己的帧,可它「看」得多勤快,跟最终质量毫无关系。真正的瓶颈是时间管理:四分之三的失败视频,是渲染开始太晚、跑得太慢或压根没启动。这不是你明天能用上的东西,但它指向一个趋势:让 AI 写代码而不是「画」视频,可能是通往可控、精确视频生成的路。
📄 原文摘要(英文)
Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a rule-consistent game or narration synchronized to on-screen events. A different route is emerging: multimodal agents that write code to render every frame and synthesize every sound. We introduce VibeVideoBench, a benchmark for evaluating agents on code-driven video generation. Its 239 self-contained tasks in English and Chinese span 12 categories, from data stories and music-driven pieces to classroom lessons and paper explainers; each asks an agent, running in its native harness inside a sandbox with no image or video generation model, to deliver a finished MP4 with sound within one hour. Evaluation is hierarchical and evidence-based: program checks of the delivery spec, about 13k task-specific rubric checks traced to the requirements they enforce, and five quality dimensions anchored per task, scored by three VLM judge panels. Across nine frontier agent systems and 2,151 scored rollouts, GPT-6 Astra and Claude Opus 5.5 rank first and second with both the judges and human annotators, and the strongest system satisfies every core requirement on 76.3% of tasks. Delivery is a second axis: counting missing videos as zero changes the ranking, and three quarters of the lost videos are final renders that started too late, ran too slowly or never started, a failure of how agents plan their hour. Agents build their own rendering pipelines and inspect their own frames, yet how much they look does not predict quality. The judges' ranking of the systems closely matches blind human preferences.