AI 拍视频终于学会用工具了
现在的 AI 生成视频，你让它拍「一个人做蛋糕」，它可能把蛋糕画成石头，或者让人的脸中途换人。这篇让 AI 学会像人一样干活：先查资料、再生成、最后自己检查一遍，不满意就重来。研究者用强化学习训练了一个「视频生成代理」，在 600 个刁钻提示词的测试里，它比原来的纯生成模型强了 19 分，而且换更强的生成工具还能继续涨分，不用重新训练。它不是你明天就能用的产品，但这是 AI 从「会画」走向「会办事」的关键一步。
📄 原文摘要(英文)
Recent advances in video generative models have enabled high-fidelity, temporally coherent video generation. However, these models often struggle to satisfy prompts requiring specialized knowledge, specific identities, physical consistency, or ordered events. In this paper, we present VideoGen-Agent, a multimodal agent trained through multitask agentic reinforcement learning to use external tools for video generation. The agent coordinates augmentation, generation, and verification tools through multi-turn interactions, using the prompt and intermediate observations to guide its decisions. We train a shared policy on a category-balanced dataset spanning six tasks. Supervised fine-tuning on teacher-generated trajectories establishes tool-use behavior, which is then refined through reinforcement learning. A category-aware hybrid reward evaluates tool-call validity, task-appropriate tool use, and generated video quality. We further introduce VABench, a held-out benchmark of 600 prompts covering procedural knowledge, single- and multi-entity identity preservation, physical consistency, scene composition, and multi-shot temporal structure. On VABench, VideoGen-Agent improves over its base text-to-video generator by 19.1 points, from 56.5 to 75.6. Upgrading the generation tools further raises the score to 86.1 without additional agent training. Human raters prefer the upgraded configuration over the strongest standalone baseline in 84.3% of comparisons. These results support learning tool use across video-generation tasks and show that the trained agent can benefit from subsequent advances in generation tools.