AI Pulse
📄 论文解读

AI看视频猜空间位置,现在能回头检查了

你让AI看一段第一视角视频,问它“桌子在椅子左边还是右边”,它只能猜——因为镜头没拍到的角度,它看不到。现有AI一次推理就下结论,错了也没法改。这篇让AI先猜一个答案,然后根据视频里的物体位置自动生成一个新视角(比如从天花板斜着看),再让AI用这个新视角重新检查自己的答案。在两项测试中,这个“先猜再查”的方法让开源模型追上了最贵的闭源模型。它不是你明天能用上的,但它告诉你:AI推理不该是一次性的,能回头检查才是真理解。

📄 原文摘要(英文)

Spatial reasoning from egocentric videos is inherently challenging because the observable evidence is constrained by the camera trajectory. Existing methods rely on single-turn inference, forcing models to resolve geometric ambiguity through semantic priors rather than verifiable evidence. We argue that spatial reasoning should be revisitable: conclusions formed under limited evidence should remain open to revision when complementary viewpoints become available. Building on this insight, we propose Reason, then Re-reason (ReRe), a training-free, inference-time framework with two phases: in the Reason Phase, an MLLM forms a spatial hypothesis from the original video; in the Re-reason Phase, it verifies or revises the hypothesis by observing a synthesized novel-view video. To enable effective cross-view revisiting, we design a Geometry-to-Video pipeline that renders strategically complementary novel views from predicted 3D geometry. These views feature an elevated, oblique perspective with scene-spanning coverage, while preserving the MLLM's native video interface without architectural modifications. Extensive evaluations on VSI-Bench and STI-Bench demonstrate that ReRe substantially boosts open-source MLLMs to rival proprietary state-of-the-art performance. Project page: https://zhenjiemao.github.io/ReRe/

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部