让机器人学会“后悔”，训练效率反超人类
机器人训练一直靠“多试错”，但现实世界试错太贵。这篇让机器人模型学会“后悔”:在关键决策点倒回去，换一条路走，对比两条路的结果，就能知道当初哪一步该改。它把世界模型从“只会往前模拟”变成“能回放重来”的引擎，而且真实机器人每做一次，模型就修正一次自己的物理判断。结果是在三个真实任务上，成功率比最强基线高约11个百分点，而且训练8小时就超过了人类平均水平——人类做不到“倒带重来”，这是机器人独有的优势。它不是你明天能用上的，但“让AI自己复盘”这个思路，可能比这次的具体成绩更值得盯。
📄 原文摘要(英文)
Action-conditioned world models offer a scalable alternative to costly real-world interaction for improving robot policies. However, existing world-model-based methods largely treat world models as forward-only surrogates of the physical environment, leaving untapped a capability no physical environment affords: rewinding to past decision points to explore counterfactual futures. We introduce Rollback, a policy improvement paradigm that turns world models from passive simulators into counterfactual engines: it rolls back to critical decision points, branches into alternative actions, and contrasts their outcomes to attribute success to individual decisions. This concentrates imagination where it matters most and distills counterfactual outcomes into fine-grained, action-level supervision without a separately learned value model. We build a scalable training system that unifies distributed world-model search with asynchronous real-robot execution, forming a self-reinforcing cycle: real-world experience continually sharpens the world model, and a sharper world model in turn drives further policy improvement. We term the world-model side of this cycle test-time physics grounding, a capability that hand-built physics simulators lack: while the world model serves as the training environment, real transitions from the policy it trains correct its dynamics through a lightweight LoRA pipeline. Experiments on three challenging real-world tasks show that Rollback improves success rates over the strongest baseline by an absolute 10.67%, 11.11%, and 11.85% on block extraction, ball tossing, and tabletop curling, respectively, while achieving more stable improvement throughout training. Although initialized from teleoperated demonstrations far below human level, Rollback surpasses the average success rate of human players on both ball tossing and tabletop curling within 8 hours of training.