让机器人听懂人话追目标：先指认再跟踪
现在的机器人视觉追踪，要么靠人预先指定目标（比如框出一个人），要么靠自然语言描述但容易在抽象空间里“猜”错。这篇论文把两件事拆开做：先让机器人从摄像头画面里选出语言描述的那个目标（比如“穿红衣服的人”），再基于这个具体的图像位置去规划跟踪路径。关键创新是加了一个滑动窗口记忆，把过去几帧选中的目标框位置也喂给模型，这样目标跑动时不会跟丢。在测试集上，单摄像头就达到了89.4%的成功率，甚至超过了一些多摄像头方案。最后在四足机器人和人形机器人上实测，也能从仿真直接迁移到真实环境。它不是你明天就能用的产品，但指明了让机器人“听懂人话并盯住目标”的一个可靠方向。
📄 原文摘要(英文)
Embodied visual tracking (EVT) requires a mobile agent to continuously follow a specific target described in natural language using only onboard vision. While recent vision-language-action (VLA) policies unify target identification and trajectory planning, their chain-of-thought (CoT) reasoning often operates in abstract spatial latents that are difficult to supervise and weakly aligned with explicit image-space detections. To address this, we introduce ReferTrack, a referring-then-tracking paradigm that grounds EVT using a single forward-facing camera. Our model first selects the target from an indexed set of bounding boxes, then decodes tracking waypoints conditioned on this image-grounded decision. To preserve target motion cues over time, ReferTrack maintains a sliding-window queue of previously selected bounding boxes, injecting their geometric features into the visual history via temporal-viewpoint-bbox indicator (TVBI) tokens. We further enhance target identification by co-training on a custom Refer-QA dataset. On EVT-Bench, ReferTrack achieves state-of-the-art single-view performance with success rates of 89.4%, 73.3%, and 74.1% on the single-target, distracted, and ambiguity tracking splits, respectively -- matching or even surpassing several multi-camera baselines on identification-heavy tasks. Finally, real-world deployments on legged and humanoid robots validate its robust sim-to-real transfer capabilities. Code is available at https://github.com/MedlarTea/referTrack.