机器人看一遍视频就会干活,不用重新训练
机器人学新任务,通常得喂大量演示数据、重新训练模型。这篇把商用视觉语言模型(GPT-6 Astra)直接当机器人的大脑:给它一段人类干活的视频,它就能看懂步骤,自己规划动作,再交给一个控制器去执行和验证——全程不更新任何模型参数。真实机器人测试里,只看人类视频就能提升任务完成率;碰到需要精确接触的活儿,再补上动作参考,效果更好。它不是你明天就能买回家的功能,但它指向一个方向:机器人以后可能像人一样,看一遍就会,而不是学一万遍才会。
📄 原文摘要(英文)
Enabling robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. No finite collection of demonstrations can cover every task and situation a robot will encounter, making the ability to learn from context at deployment essential for generalization. Such in-context learning (ICL), however, remains largely beyond the reach of existing robotic policies. The broad agentic capabilities of commercial vision-language models (VLMs), such as GPT-6 Astra, raise a compelling question: can these models learn from demonstrations, examples, and interaction feedback, then translate that information into executable and verifiable robot behavior from a new initial state without gradient updates or persistent changes to task-specific parameters? We introduce GPT-Policy, a general-agent framework for in-context robot learning. GPT-Policy integrates a context compiler that preserves task-relevant visual transitions, a VLM that proposes robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome. We evaluate its reliability and limitations through task success and efficiency metrics, matched comparisons across models, and controlled context ablations. In real-robot trials, human video demonstrations improve task completion even without robot action labels, while aligned action references yield further gains on contact-sensitive tasks. These findings position GPT-Policy as a step toward robot adaptation through in-context learning, providing an empirical foundation for translating the general-purpose capabilities of VLMs into physical behavior and clarifying the challenges that must be overcome for reliable deployment.