机器人不再需要先“想象未来”再动手
现在的机器人控制，主流做法是让模型先“脑补”出未来几秒的画面，再根据画面决定动作。这很贵，而且画面一旦猜错，动作也跟着错。这篇论文反着来：把视频扩散模型“预测未来”的能力，蒸馏成一个只看当前画面就能直接输出动作的编码器。相当于把“预判”练成了肌肉记忆，不再每次现算。结果是在真实和模拟任务上，成功率超过目前最强的视觉-语言-动作模型，而且能做到 11Hz 的实时控制，背后还是个 140 亿参数的大模型。它还能从纯视频演示里学新任务，连不同机器人的视频都行，不需要动作标注。这不是你明天能用的东西，但它指向一个趋势：机器人控制正在从“生成未来”转向“直接感知”，更省、更快、更稳。
📄 原文摘要(英文)
World Action Models (WAMs) use the predictive knowledge of pretrained video diffusion models for robot control through future video prediction. However, iterative video denoising is costly, and inaccurate visual predictions can compromise action prediction. Our key idea is to make this predictive knowledge available through visual representations computed from the current observation, enabling action prediction without generating future videos. We introduce DreamFree, a WAM that repurposes a pretrained video diffusion model as a predictive visual encoder through representation distillation. The visual encoder learns to predict a frozen video diffusion model's representation using only the current observation. An action module is then trained to predict actions by extracting action-relevant information from the visual encoder. Across real-world and simulated tasks, DreamFree achieves higher success rates than state-of-the-art VLAs and WAMs, while performing real-time closed-loop control at 11Hz with a 14B video diffusion backbone. Experiments show that DreamFree can translate stronger video priors and additional video data into better robot control and enable learning new robot tasks from video demonstrations, including videos from a different embodiment, without action annotation for the new tasks.