机器人学新技能，不用再花几百小时录数据了
教机器人一个新动作，现在主流做法是让人远程操控它做几百上千次，录下数据再训练——光录数据就要几百小时人工。这篇换了个思路：让机器人自己先试。它给大模型装了一个「视觉语言模型」当大脑，把「把杯子放到桌上」这种长任务拆成几步短动作，然后让机器人自己去探索、试错、收集数据，再用这些数据微调策略，最后用强化学习收尾。在多个任务上，它用更少的数据就超过了现有方法。它不是你明天就能在自家机器人上跑的东西，但指向一个方向：机器人学新技能，可以不再靠人手把手教。
📄 原文摘要(英文)
How to efficiently finetune robot policies to learn new tasks on the fly? State of the art robotic manipulation policies are based on behaviour cloning of large vision-language-action (VLA) models with billions of parameters on huge teleoperation datasets. While this simple approach has enabled significant advances for robotic manipulation, finetuning of VLA policies for learning new tasks still remains an open problem. In particular, collecting teleoperation datasets requires hundreds of hours of expensive human labour and the alternative, reinforcement learning (RL), can be notoriously sample-inefficient especially for long-horizon tasks. In addition, RL with VLAs imposes several challenges due to the model's size and architectural design. In this work, we propose EXIMO, an efficient algorithm for finetuning of VLA policies. EXIMO operates in three stages: explore, imitate, and optimize. During the explore phase, EXIMO equips the VLA with a vision language model (VLM) that acts as a planner. The VLM thinks and breaks down challenging long-horizon problems into shorter ones for the VLA. The VLM, together with the VLA, is used to collect an orchestrated dataset on new tasks. During the imitate phase, the VLA is finetuned with the orchestrated data. Finally, during the optimize stage, we use residual off-policy RL to further finetune the policy. In our experiments, we ablate all three stages of EXIMO and show that it outperforms existing approaches significantly in terms of sample-efficiency and final performance.