给机器人编程的AI，考试及格率只有34%
给机器人写代码的AI，第一次有了统一的“高考”:1,156道题、35套卷、47种机器人，从抓取物品到拼图、多机器人协作、全身动态控制。最强AI(OpenAI的GPT-6 Astra)在模拟真实世界的标准考场上，及格率只有34%。更关键的是，这场考试允许AI自己选答题方式——直接控制、写代码控制、还是训练一个策略——而不是像以前的机器人测试那样只考“看画面动手”。它不是你明天能用的东西，但它划了一条线：在AI进物理世界之前，先得过了这关。
📄 原文摘要(英文)
Early successes on agentic robotics point to a potential paradigm shift toward using coding agents to control robots. Existing robotics benchmarks are built around visuomotor policy learning, leaving it open whether coding agents can reliably control diverse robots under physical-world constraints. We therefore introduce the Embodied First Exam, a unified, comprehensive, and agent-native evaluation framework where coding agents can choose their own control approach: direct control, code to control, or train a policy to control. We hope it serves as a first exam for coding agents before they enter the physical world, with 1,156 tasks in 35 suites on 47 robots, including 9 new suites that go beyond pick-and-place to spatial puzzles, dexterous manipulation, multi-robot interaction, dynamic whole-body control, and sim-to-sim engineering. Each task can be run in up to three evaluation settings. The Standard Setting mimics the real world: the agent receives only sensor observations and, by default, cannot reset the environment. Its Realtime variant keeps the environment running while the agent reasons. The Privileged Setting gives the agent access to the full simulator state and unlimited resets, allowing it to search for an oracle solution for training purposes. Success rates in this setting serve as an approximate upper bound. On the 456 core tasks, the strongest of eight agents, GPT-6 Astra in Codex, achieves a success rate of 34% in the Standard Setting, leaving much room for improvement. To see where agents fail, we label the stage at which failed attempts go wrong and compare failures across agents, settings, and stage categories.