AI Pulse
📄 论文解读

AI看得见眼前,却记不住身后

我们默认AI看视频就是“看见”,但真正的空间感知是:眼前的东西移出画面后,你还记不记得它、能不能预判它下一步在哪、能不能用它来行动。这篇论文把十个主流视觉模型放在一个“边走边看”的模拟环境里,测了四层能力:场景结构、物体持久性、预测性更新、以及能否指导行动。结果很反直觉:一个模型能把当前画面读得很准,不代表它能记住刚离开视野的物体;能预测未来画面,也不代表这个预测能用来指导行动。研究者还做了对照实验,证明“读得准”和“真的在用历史信息”是两回事。它不是你明天能用上的东西,但它提醒我们:AI的“空间感”可能只是对单张图片的聪明解读,离真正的世界模型还很远。

📄 原文摘要(英文)

An observer sees only part of the surrounding world, yet must keep track of what lies beyond the current view. Spatial perception therefore involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action. Do visual foundation models provide representations that support such a state? We study this question through four levels of spatial capability---spatial structure, object persistence, predictive spatial update, and actionability---using frozen-feature probes on ten representations under streaming egocentric observations of static indoor environments. The resulting capability profiles vary across tasks, network layers, and prefix lengths: strong readout of the visible scene does not ensure accurate object persistence or future-feature retrieval. Matched controls separate readout accuracy from reliance on visual history and motion. Native future prediction retains recoverable spatial structure, but a spatial reader trained on real features does not transfer to predicted features unchanged. Finally, goal-dependent candidate selection does not establish reliable continuous action utility: predicted actions respond to goal changes, yet supplying the correct goal does not reliably improve progress. Across the four levels, object persistence and actionability build on camera-pose readout, whereas feature prediction builds on visible-scene readout. Together, these findings motivate evaluating spatial representations not only by what can be decoded from them, but by whether that information is maintained beyond observation, updated predictively, and used for goal-directed action.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新