AI Pulse
📄 论文解读

让机器人用「后见之明」教自己

机器人学新任务,通常只有两条路:要么照着专家示范死记,要么靠试错拿奖励——前者学不会自己犯错后怎么救,后者奖励太稀疏、根本不知道哪一步做对了。这篇让模型用「未来的视频」当老师:训练时,模型先按自己的方式一步步生成动作,走到哪一步,就让同一个模型偷偷看一眼这段演示「后来发生了什么」,据此给出「这一步本该怎么做」的完整动作分布,再回头把学生拉向这个答案。学生部署时看不到未来,只带正常输入,也不需要奖励或仿真器。它把「事后诸葛亮」变成了密集的监督信号。对做机器人操作、想让预训练模型可靠落地具体任务的人,这是值得盯的路线;对普通人,它意味着机器人离「看一遍就能学会、还能自己纠错」近了一小步。

📄 原文摘要(英文)

World action models (WAMs) have shown promising generalization and data efficiency in robot manipulation by coupling action generation with predictions of future visual dynamics. Translating these capabilities into reliable task execution, however, remains a challenge for existing post-training methods. Offline supervised fine-tuning provides dense supervision on expert trajectories, but its flow-matching action expert is trained only on noised demonstration actions and never on the intermediate states produced by its own sampler; task-specific adaptation can also compromise pretrained capabilities. On-policy reinforcement learning trains on the policy's own behavior, yet sparse task rewards make exploration and credit assignment difficult, while dense reward shaping still provides scalar evaluations rather than direct action targets. We introduce HINT-WAM, the first on-policy self-distillation framework that supervises the action expert on its own sampling trajectories using hindsight, the future video of the demonstration, as privileged information available to the WAM itself. For each demonstration observation, the student generates its action through its own denoising trajectory, and at the states along this trajectory the same model acts as a teacher conditioned additionally on this hindsight, providing a conditional action distribution as a dense training target, while the student sees only deployment-time inputs and needs no environmental rewards or simulator interaction. To transfer this supervision to a flow-matching action expert, HINT-WAM distills the teacher into the student with an energy score, a distribution-level objective that matches the teacher's action distribution from the student's own samples rather than regressing its velocity field. The project web is available in https://tuandattt.github.io/hintwam/.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新