给机器人看视频，比手把手教更管用
教机器人干新活，通常得给它大量带动作标注的演示——这很贵。这篇发现，光给它看新任务的视频、不标动作，成功率就能从 25% 跳到 56%;每类任务再补一条带动作的示范，直接到 71%。也就是说，机器人看多了「别人怎么做」，自己就能反推出动作，少量示范只是帮它校准。基于这个发现做的 WorldBridge，把「看视频」和「出动作」拆成两个专家，用注意力机制连起来，在多个操作测试里刷新了纪录。它不是你明天能用上的，但给机器人行业指了条省钱的路：与其录海量带标注的数据，不如多拍视频。
📄 原文摘要(英文)
Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge. By varying video and action supervision for tasks outside an initial training set, we examine their respective contributions to task transfer. On RoboTwin 2.0, using only in-distribution video-action supervision yields a 25.07% success rate on out-of-distribution tasks. Introducing videos of these tasks without their action labels raises success to 55.90%, while adding just one action-labeled demonstration per task further improves it to 70.70%. These findings suggest that broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, while even sparse action supervision further strengthens those capabilities. Guided by this perspective, we introduce WorldBridge, a WAM with separate video and action experts connected through structured attention and trained with joint flow matching. WorldBridge incorporates visual memory at two temporal scales and action forcing, alongside a unified data processing pipeline that standardizes and filters heterogeneous robot trajectories and egocentric hand-derived supervision. Across multiple manipulation benchmarks, WorldBridge achieves state-of-the-art performance among the compared methods, including 96.23% on RoboTwin 2.0 full data setting and 80.9% on clean to random setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. Our study highlights the complementary roles of video prediction and action grounding in developing robots that generalize across tasks.