让AI写代码来开机器人，它输在手上没输在脑子上
机器人操控的排行榜上，现在冲进来一个「程序员」:它不学动作，而是像修bug一样，看摄像头画面、写小工具、失败就改代码，硬是把双手机器人开到了新纪录。但真正有意思的是它卡在哪：推理、找东西、记忆都行，成功率能到九成；一碰到「插进去」这种需要手感的活，直接掉到8%。它看得见自己为什么失败，却使不上劲——瓶颈不在想，在手。这套系统只用了每个任务10个场景练出来的工具，换10个新场景照样用，代码即经验。它不是你明天能用的东西，但它指了个方向：让AI像程序员一样积累技能，而不是像运动员一样练肌肉。
📄 原文摘要(英文)
Agents built on vision language models (VLMs) now compete with learned policies on robot manipulation leaderboards. We ask how far a coding agent can go when it controls a robot instead of a computer, and where it stops. Our system, HexaShell, keeps the harness thin: it only turns the agent's commands into robot motion. The agent drives a robot with two arms through one command line. It sees three RGB cameras and knows their calibration. After each failed attempt, the agent reads the log, finds the cause, and writes or fixes small programs called tools. In this way it keeps what it learns as code, as a coding agent does when it fixes a failing test. The tools may use only what the robot observes. They use triangulation to find where objects are, relative to each other and to the gripper. If two cameras see the same point, the point lies where the two camera rays meet. If a point lies on a known plane, such as the table or the top of a block, it lies where its ray meets that plane. When the head camera is not enough, the robot moves a wrist camera to get a second view. We build the tools on only ten layouts of one seed per task, then freeze them and test them on ten new layouts of the same seed. We use Codex with GPT-6-Astra at medium reasoning effort. On all 42 RoboDojo tasks, HexaShell reaches a success rate of 39.7% and a progress score of 48.0, averaged in the same way as the leaderboard. This is a new state of the art on RoboDojo. The limit is contact, not reasoning: pushing reaches 92.5% and memory tasks 65.0%, but tight insertion only 8.3%. When the agent fails, it usually sees the failure but cannot make the motion work. We open source the tools evolved so far at https://github.com/HexaFuture/HexaShell.