机器人不该只看AI的“最终答案”
现在的机器人视觉系统，都是先让一个“眼睛”把画面压缩成摘要，再拿摘要去决定动作。但抓取、插孔这类精细操作，靠的恰恰是摘要里最容易丢掉的细节：边缘、深度、物体状态的微小变化。这篇论文干脆拆掉那个固定“眼睛”，让画面信息在神经网络里和语言指令一起流动，动作决策可以随时从不同深度取用视觉信息。结果很反直觉：动作真正依赖的关键视觉信号，往往在网络的早期层就出现了，而不是等它被压缩成最终表示之后。所以，让机器人直接接触“计算过程中的画面”，比只给它“最终答案”更稳、更抗干扰。这不是你明天就能用上的技术，但它指向一个趋势：给机器人做视觉，可能不该再沿用“先看懂、再行动”的老套路。
📄 原文摘要(英文)
Vision-language-action models are typically built on a simple premise: perception is first compressed by visual encoders, and action reasoning begins only afterwards. Yet manipulation often hinges on precisely the visual signals most vulnerable to this abstraction, e.g., local geometry, boundaries, depth, and subtle state changes. This raises a fundamental question: should action policies inherit a fixed visual endpoint, or directly access evolving visual computation? We introduce NEO-VLA, a structurally encoder-free vision-language-action framework. Images are tokenized by lightweight patch convolutional layers and evolve jointly with language throughout the backbone, allowing a flow-matching action expert to draw from instruction-conditioned visual representations at different depths. This simple change reveals a consistent pattern. Native architectures outperform encoder-based counterparts under controlled comparisons; full fine-tuning reduces but does not erase the gap, and additional scale alone cannot recover it. More strikingly, layer-wise probing shows that the action-critical signals often peak well before the final visual representation. Across diverse manipulation settings and distribution shifts, NEO-VLA consequently achieves stronger and more robust control. Together, these results suggest that embodied control benefits from access to visual representations across the computational hierarchy, rather than a fixed encoder output.