机器人看人视频就能干没学过的活
机器人学新任务，通常得先喂几千条它自己的演示数据。这篇换了个思路：像大模型靠上下文提示就能答题一样，机器人只看一段人类干活的视频，就能在没练过的任务上直接上手。研究者造了个模型，把人类视频当“任务说明书”，再配了个自动生成的数据集——把机器人轨迹转成语义匹配的人类视频，共8.6千个任务、7.4万对样本。在模拟的7个全新任务上，成功率47%，比最强的视频动作基线高出29.5个百分点；真实世界里，它也能跟着人类视频，完成多物体场景、长时序操作和精细插孔这类没见过的配置。它不是你明天就能买回家的东西，但方向很明确：以后教机器人干活，可能不用写代码、不用录机器人演示，拍段人干活的视频就够了。
📄 原文摘要(英文)
Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in robot learning. In large language models, a novel task can be performed simply by specifying it in the context, without any parameter update. This form of in-context learning (ICL) turns generalization into a problem of task specification. To achieve cross-task generalization, we bring this paradigm to robotic manipulation, and argue that the natural task specification for manipulation is a human video: unlike language, it provides rich visual cues about the intended task evolution. We present Zero-WAM, a causal video-action model that executes unseen tasks by following in-context human video guidance. To address the scarcity of task-rich paired human-robot data, we propose an automatic pipeline that converts task-sampled robot trajectories into semantically matched human videos, yielding HumanGen, a dataset of 74.2K human-robot ICL pairs across 8.6K tasks. For model training, we further introduce an in-context future chunk prediction (IFP) objective that suppresses shortcuts learned from seen tasks and forces the policy to draw task information from the video prompt. On seven unseen tasks in RoboTwin 2.0 simulation, Zero-WAM achieves a 47.0% average success rate, an absolute improvement of 29.5 percentage points over the strongest video-action baseline. In real-world evaluations, it follows human video guidance to generalize to unseen task configurations involving multi-object scenes, long-horizon manipulation, and fine-grained insertion.