给机器人装上“记忆”，只看一帧也能干完活
现在的机器人大模型大多“看完一张照片就动手”，做完一步就忘了之前看过什么。这篇给模型加了个“记忆流”:把每一帧画面和指令捆成一个时间单元，单元内部可以互相看、单元之间按顺序传递，于是机器人能边做边回忆，而且不增加任何参数量。训练时还故意让画面间隔忽长忽短，让机器人不怕实际运行时的延迟抖动。在真实机器人和模拟环境上都超过了当前最强的 pi0.5，尤其是需要记忆和精细定位的任务。它不是你明天就能装进自家机器人用的，但这是“机器人看连续画面干活”从玩具走向可靠的一大步。
📄 原文摘要(英文)
Vision-Language-Action (VLA) models have demonstrated effectiveness in robot manipulation, yet state-of-the-art models such as pi0.5 operate under a single-frame paradigm, limiting their ability to retain past observations and develop precise spatial perception. In this paper, we propose StreamPI, a streaming multimodal temporal modeling framework that equips single-frame VLA with temporal reasoning capability without introducing any additional parameters. One core design is instruction-anchored temporal modeling. It treats each (visual observation, language instruction) pair as an atomic temporal unit: bidirectional attention within each pair enables cross-modal fusion, while causal attention across pairs preserves autoregressive streaming inference. This ensures the language instruction serves as a persistent semantic anchor throughout task execution. To bridge the gap between synchronous training and asynchronous real-robot deployment, we introduce a andom-interval streaming training strategy: a proper inter-frame interval (e.g., every 3 frames) enables faster and smoother action execution. Beyond this, randomizing the interval further improves robustness to frame-timing perturbations, supporting asynchronous deployment in practice. Furthermore, by leveraging the length extrapolation capability of the LLM backbone, StreamPI seamlessly inherits pretrained single-frame weights and supports flexible single-frame and multi-frame inference. Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.