AI Pulse
📄 论文解读

AI看视频的空间感,连人类一半都不到

人类看一段视频,能在大脑里拼出一张俯视图,知道哪里是哪里;最强的AI模型却做不到。新基准GST-Bench用6790分钟合成视频考了22个顶尖视觉模型,人类得分79,最强模型只有42.68。更关键的是,模型在局部空间理解上并不差,差的是把长时间看到的画面整合成一张连贯地图的能力。这不是它明天能帮你导航或看监控的差距,而是告诉你:AI对世界的理解,还停留在“看哪是哪”,没到“看完知道全局”。

📄 原文摘要(英文)

Spatial intelligence is fundamental to embodied agents, yet existing benchmarks focus on local spatial perception from single or few viewpoints, overlooking global spatial awareness over continuous, long-horizon visual streams. To address this limitation, we introduce the Global-Spatial-Temporal Benchmark (GST-Bench), a VQA benchmark for global spatial intelligence in video understanding, comprising human-verified questions derived from 6,790 minutes of synthetically generated video. It requires models to perform accurate spatial inference from novel viewpoints unseen in the input video and to map egocentric observations onto global top-down images. A comprehensive evaluation of 22 state-of-the-art VLMs exposes a striking gap between models and humans: the strongest zero-shot model attains only 42.68, far below the human score of 79.08. To probe the cause of this gap, we construct GST-Bench-Local and find that models, despite strong local spatial understanding under the same task formulation, still fail to consolidate long-horizon observations into a globally consistent scene representation. We further provide GST-Train, a dataset for global spatial reasoning, as a complementary resource to facilitate future research on this challenge.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新