让AI自己写代码开机器人,成功率100%
现在的机器人干活,基本是工程师提前写好每一步动作,换个场景就废。这篇反着来:让通用AI像人一样看着任务、自己写程序、自己发指令,干砸了还能自己改。在真实机器人上,它靠视觉判断积木位置,现场写代码控制机械臂,三种摆放构型成功率100%、100%、80%,还能叠毛巾、扔东西、从视频里学装配。它不是你明天能买回家的产品,但指向一个方向:机器人不再需要为每个任务单独编程,AI自己就是那个程序。
📄 原文摘要(英文)
We demonstrate that a general-purpose agent can directly drive a physical robot throughout task execution without any task-specific or environment-specific training. We introduce Agent as Policy (AGP), which places task planning and execution under the agent's control. Given a task and a robot interface, the agent interprets visual evidence, writes executable programs, issues motion commands, and revises its actions in response to physical outcomes. This brings the agent's reasoning and programming capabilities into continuous interaction with the physical world. We study AGP across multiple real-world manipulation tasks spanning precision manipulation, dynamic motions, and deformable objects. These include assembly from human videos, block construction from goal images, die reorientation, targeted throwing, and bimanual towel folding. AGP achieves success rates of 100%, 100%, and 80% on three block construction configurations. These findings establish a path for general-purpose agents to act as robotic policies, extending their autonomy to physical manipulation through runtime reasoning, programming, and interaction.