AI Pulse
📄 论文解读

AI视频模型不再只做特效,开始学物理常识

现在的AI视频生成模型(如Sora)擅长创造炫酷画面,但不懂物理——比如机器人抓杯子,它可能只关心杯子好不好看,却不知道抓的力度和角度。这篇论文反其道而行:用混合专家架构(MoE)让模型在生成视频时兼顾效率和物理合理性,并专门加入大量机器人操作、导航的第一人称视频数据,让模型学会“动作如何影响世界”。他们还设计了一套奖励系统,不仅看画面美不美,还检查动作是否符合物理规律、任务是否完成。结果:模型生成的机器人操作视频更真实,能直接用于训练机器人。这不是你明天能用的工具,但它让AI从“画师”向“物理世界模拟器”迈了一步。

📄 原文摘要(英文)

Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新