机器人看人类视频学新技能，成功率翻倍
机器人学新技能，通常得先看大量机器人自己的演示。这篇让机器人从「看人类视频」直接学会做没练过的任务：它把「预测未来画面」这件事拆成一步完成，再用真实未来画面当老师、自我蒸馏，让预测更准；同时把历史画面也喂进去，让模型知道上下文。结果：人类视频里演示过、但机器人训练数据里从没出现过的任务，成功率从最好的同类模型的 19.4% 提到 50.0%。这不是明天就能用的东西，但它指向一个方向：机器人或许能像人一样，看别人做一遍就会了。
📄 原文摘要(英文)
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self-Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self-distillation to predict future representations from noise, using representations extracted from ground-truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real-world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long-horizon and out-of-domain tasks under zero-shot environment generalization, and substantially improves human-to-robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.