让大模型直接开机器人：零训练，看一眼就会
现在的机器人控制通常要针对每个任务专门训练，换个场景就废。这篇论文反着来：不训练机器人，而是把大模型（VLM）当成大脑，给它一个极简的遥控器——只有平移、旋转、抓取三组离散指令。它像人一样看画面、想下一步、执行、再看结果调整。零样本时它已经超过多个专门训练过的基线；给一条演示后，成功率从53.2%涨到73.6%，超过强基线π0.5的46.0%。同样的框架还能直接搬到真实机械臂上做抓取和堆叠。它不是你明天就能用的产品，但指向一个方向：机器人可能不再需要为每个任务单独训练，而是靠通用大模型的常识+几条演示就能上手。
📄 原文摘要(英文)
Humans can seamlessly adapt to both physical and digital worlds, suggesting that while a digital-to-real gap exists in embodiment, environment and task, human intelligence itself may transfer across this gap. This naturally raises a fundamental question: can the intelligence of vision-language models (VLMs) similarly generalize from the digital world to the physical world for robotic control? We investigate this question through RoboDawn, a human-intuitive interface that exposes robotic control to an agentic VLM through a compact set of discrete translation, rotation, and gripper commands. Using this interface, the VLM controls a robot in a closed loop: it observes the current visual state, reasons about the next action, executes it, and adapts subsequent decisions to the resulting state. Furthermore, we introduce an in-context learning (ICL) scheme that uses a few demonstrations to ground the VLM in both interface usage and task-solving strategies. Experiments on RoboTwin 2.0 C2R and RoboDojo demonstrate that RoboDawn achieves strong performance without task-specific robot training. In the zero-shot setting, RoboDawn outperforms several strong policies trained on benchmarkspecific robot data, while a single in-context demonstration further yields substantial performance gains and establishes state-of-the-art (SOTA) results. On RoboTwin 2.0 C2R, the success rate increases from 53.2% zero-shot to 73.6% one-shot, exceeding the solid baseline π0.5 (46.0%). Similar gains are observed on RoboDojo, where success rate improves from 35.67% zero-shot to 47.17% one-shot. The same framework also transfers to real-world robots, performing block-in-basket and block stacking on Franka.