AI Pulse
📄 论文解读

让AI先学会“看后果”,再学“做动作”,反而更会做事

现在的机器人/具身AI训练,通常是直接教它“看到什么就做什么动作”。这篇论文反着来:先让AI学会“如果我做了这个动作,画面会变成什么样”,也就是预测后果,然后再去学动作本身。结果发现,先学会预测后果的AI,在换场景时表现更稳、泛化更好;反过来,学动作的过程也会让AI对画面变化的预测更准。两者互相喂,比单练一边都强。它不是你明天就能用上的东西,但它指向一个更接近人类学习方式的训练思路:先理解世界怎么变,再决定自己怎么做。

📄 原文摘要(英文)

World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新