机器人看一遍视频就会干活了
机器人学新任务，通常得先喂几百上千条演示数据，换个环境就抓瞎。这篇把商用视觉大模型(比如 GPT-6 Astra)直接当成机器人的“大脑”:给它一段人类干活的视频，它就能看懂步骤，自己规划动作，再交给一个控制器去执行和验证——全程不用重新训练模型，不用改任何参数。真实机器人测试里，光看人类视频就能提升任务完成率；如果视频里还带动作标注，对需要精确接触的任务(比如插拔、拧螺丝)效果更好。它不是你明天就能买回家的功能，但它指向一个方向：未来机器人可能像人一样，“看一遍就会”。
📄 原文摘要(英文)
Enabling robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. No finite collection of demonstrations can cover every task and situation a robot will encounter, making the ability to learn from context at deployment essential for generalization. Such in-context learning (ICL), however, remains largely beyond the reach of existing robotic policies. The broad agentic capabilities of commercial vision-language models (VLMs), such as GPT-6 Astra, raise a compelling question: can these models learn from demonstrations, examples, and interaction feedback, then translate that information into executable and verifiable robot behavior from a new initial state without gradient updates or persistent changes to task-specific parameters? We introduce GPT-Policy, a general-agent framework for in-context robot learning. GPT-Policy integrates a context compiler that preserves task-relevant visual transitions, a VLM that proposes robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome. We evaluate its reliability and limitations through task success and efficiency metrics, matched comparisons across models, and controlled context ablations. In real-robot trials, human video demonstrations improve task completion even without robot action labels, while aligned action references yield further gains on contact-sensitive tasks. These findings position GPT-Policy as a step toward robot adaptation through in-context learning, providing an empirical foundation for translating the general-purpose capabilities of VLMs into physical behavior and clarifying the challenges that must be overcome for reliable deployment.