机器人补上空间感这一课
机器人一直靠 2D 视觉预训练「看懂」世界，却不太懂「离我多远、朝哪边」。这篇的做法是：让模型在训练时不断把自己的视觉表征对齐到一个冻结的 3D 感知模型的空间表征上，把空间感当成一项可迁移的基础能力，在大量不同机器人、任务、场景的数据上持续预训练，而不是等到具体任务才临时补。为了吃下这些杂七杂八的数据，它用一组随特征自适应的门控，从模型多层视觉表征里挑着聚合去做空间监督；老师模型还能利用只有部分样本才有的深度和相机标定，把零散的几何标注也用起来；训练时随机丢掉状态信息，逼模型多看画面、少背轨迹。所有辅助部件只在训练时存在，推理时原样复用预训练权重、架构不变。它不是你明天能用上的东西，但方向是：以后的家用或仓储机器人，可能不用为每个新场景重新教一遍「东西在哪」。
📄 原文摘要(英文)
Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation. Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience. We present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training. Initialized from a pretrained pi05 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model. We curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes. To accommodate this diversity, we introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision. The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations. Probabilistic state dropout encourages reliance on visual cues and mitigates overfitting to demonstration trajectories. All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture.