AI Pulse
📄 论文解读

视频生成模型不再只做视频,开始学物理

现在的视频生成模型(如Sora)擅长创造好看的画面,但不懂物理规律——比如机器人抓杯子,它可能生成一个杯子飘在空中的画面。这篇论文反其道而行之:用混合专家架构(MoE)让模型在生成视频时兼顾效率和物理合理性,并专门加入大量机器人操作、导航的第一人称视频数据,让模型学会“动作导致结果”的因果逻辑。他们还设计了一套奖励系统,不只检查画面美不美,还检查动作是否符合物理、任务是否完成。最终模型LingBot-Video在机器人模拟任务中表现更好。它不是你明天能用上的工具,但指明了方向:未来的视频生成模型可能不只是内容创作工具,更是机器人的“物理世界模拟器”。

📄 原文摘要(英文)

Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部