让AI先学会“看后果”,再学“做动作”,反而更会做事
现在的机器人/具身AI训练,通常是直接教它“看到什么就做什么动作”。这篇论文反着来:先让AI学会“如果我做了这个动作,画面会变成什么样”,也就是预测后果,然后再去学动作本身。结果发现,先学会预测后果的AI,在换场景时表现更稳、泛化更好;反过来,学动作的过程也会让AI对画面变化的预测更准。两者互相喂,比单练一边都强。它不是你明天就能用上的东西,但它指向一个更接近人类学习方式的训练思路:先理解世界怎么变,再决定自己怎么做。
📄 原文摘要(英文)
World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.