机器人一换环境就失灵？问题出在它偷看了不该看的东西
机器人学动作时有个坏习惯：它会把画面里跟任务无关的细节(比如背景颜色、光照)当成线索来用。训练时这些线索恰好和正确动作相关，它就这么蒙混过关了；可一到新环境，线索变了，它就抓瞎。这篇论文的做法很直接：第一步，先不给它看画面，只靠语言指令和机械臂末端要到达的位置来学动作；第二步，再让画面信息通过一个专门的“接口”进入，而这个接口被强制要求只保留和到达目标位置有关的空间信息。相当于给视觉信息装了个过滤器，逼它只看该看的。在多个主流机器人模型上，这套方法让成功率在新环境下提升了 4 到 17 个百分点。它不是你明天就能用上的东西，但它指出了机器人泛化能力差的一个被忽视的根源，并给出了一个不挑模型的通用解法。
📄 原文摘要(英文)
Robot foundation models achieve strong in-distribution performance but often degrade under visual distribution shifts. When learning to generate actions from pretrained visual representations, models may exploit task-irrelevant visual cues that correlate with demonstrated actions within the training distribution. Such vision-action shortcuts can undermine generalization when these correlations change under distribution shifts. Mitigating these shortcuts requires constraining how visual information is used for action generation while preserving task-relevant spatial information. We propose Latent Interface Training (LIT), a framework-agnostic two-stage strategy that first establishes a spatial-goal-conditioned action prior without images, then constrains visual conditioning through a pose-supervised latent interface. Stage 1 trains the action expert to generate action chunks conditioned on language, robot state, and each demonstrated chunk's terminal SE(3) end-effector pose, learning goal-directed action generation independently of visual cues. Stage 2 introduces a latent interface that aggregates visual and semantic representations and serves as the pretrained action expert's only visual conditioning pathway. The interface is supervised to reconstruct the terminal pose previously used to condition Stage 1, encouraging it to retain the goal-relevant spatial information needed for action generation. Across four vision-language-action and world-action architectures (Pi0.5, MolmoAct2, FAST-WAM, and ImageWAM), LIT improves overall LIBERO-Plus success by 3.87-10.70 percentage points while preserving or improving average LIBERO success. Real-world evaluations show 13.30-16.70 percentage-point gains in success aggregated across three tasks under unseen camera configurations, lighting variations, and distractors.