AI学会看视频推理,不用文字也能懂物理
现在的AI大多靠文字+图片学推理,但人类看世界是直接看画面、不用每步都翻译成语言。这篇让AI纯靠看视频学复杂推理:它自己看一堆演示视频,然后像人一样在脑子里“演”一遍,判断物理对不对、逻辑顺不顺,全程不依赖文字描述。在16种任务上,它比同类模型强了25%,甚至能看懂物体怎么动、长期计划怎么走。它不是你明天能用上的,但方向很明确:AI离“看懂世界”又近了一步。
📄 原文摘要(英文)
Learning broad world knowledge directly from raw visual data is a fundamental capability of intelligence. We introduce UniVR, the first investigation into simultaneously learning complex reasoning, fine-grained physical dynamics, and long-term planning from pure visual demonstrations. At its core, UniVR features VR-GRPO, a reinforcement learning paradigm with complementary global and step-level rewards. This approach enforces logical coherence and physical consistency throughout the reasoning process without requiring task-specific heuristics or image-text pairs. To train and evaluate UniVR, we construct VR-X, a large-scale benchmark curated from 16 diverse sources spanning long-horizon manipulation, spatial puzzles, and physical reasoning. It is the first comprehensive suite to assess these heterogeneous capabilities under a purely visual protocol. Remarkably, UniVR achieves up to a 25% improvement on VR-X, and its superior visual reasoning also boosts performance on various multimodal understanding benchmarks. These findings underscore the vast potential of reasoning within visual spaces, with all code, data, and models are open-sourced for further research.