不训练机器人，直接让大模型上手干活
让机器人学会新任务，通常要专门训练、喂数据。这篇反着来：把机器人的动作简化成几个离散指令(平移、旋转、夹爪)，然后直接让现成的视觉语言大模型像人一样「看一眼、想一步、动一下」，在真实环境里闭环操作。零样本时它已经超过不少专门训练过的模型；给一条示范后，成功率从 53.2% 涨到 73.6%，还超过了强基线。它不是你明天能用的东西，但它指向一个方向：机器人可能不需要为每个任务重新训练，大模型本身的理解力就能直接迁移到物理世界。
📄 原文摘要(英文)
Humans can seamlessly adapt to both physical and digital worlds, suggesting that while a digital-to-real gap exists in embodiment, environment and task, human intelligence itself may transfer across this gap. This naturally raises a fundamental question: can the intelligence of vision-language models (VLMs) similarly generalize from the digital world to the physical world for robotic control? We investigate this question through RoboDawn, a human-intuitive interface that exposes robotic control to an agentic VLM through a compact set of discrete translation, rotation, and gripper commands. Using this interface, the VLM controls a robot in a closed loop: it observes the current visual state, reasons about the next action, executes it, and adapts subsequent decisions to the resulting state. Furthermore, we introduce an in-context learning (ICL) scheme that uses a few demonstrations to ground the VLM in both interface usage and task-solving strategies. Experiments on RoboTwin 2.0 C2R and RoboDojo demonstrate that RoboDawn achieves strong performance without task-specific robot training. In the zero-shot setting, RoboDawn outperforms several strong policies trained on benchmarkspecific robot data, while a single in-context demonstration further yields substantial performance gains and establishes state-of-the-art (SOTA) results. On RoboTwin 2.0 C2R, the success rate increases from 53.2% zero-shot to 73.6% one-shot, exceeding the solid baseline π0.5 (46.0%). Similar gains are observed on RoboDojo, where success rate improves from 35.67% zero-shot to 47.17% one-shot. The same framework also transfers to real-world robots, performing block-in-basket and block stacking on Franka.