AI Pulse
📄 论文解读

给AI装上工具箱和记事本,它才真会干活

光让AI多“想”一会儿,解决不了需要查证和计算的问题——在22个基准测试上,单纯增加推理预算只带来微薄提升。研究者给多模态大模型配了一个叫OpenVLHarness的外挂:一层是工具箱,能调用各种外部工具(比如测量、搜索),并把结果转成模型能读懂的图文证据;另一层是持久记忆,把查到的图片、数据、网页都存下来,供后续推理随时取用。模型还能写代码对这些存下来的证据做计算。装上这套装备后,在22个基准上平均提升9.7到12.6分,而且用中等推理强度的模型,就能超过最强模型开足马力的表现,token成本还不到对方四成。它不是你明天就能用上的产品,但指向一个明确趋势:下一代AI的差距,不在“想得多深”,而在“能调用什么、记得住什么”。

📄 原文摘要(英文)

Multimodal large language models (MLLMs) have made rapid progress on visual reasoning, yet many tasks require evidence that is difficult to obtain through internal reasoning alone: our controlled study on 22 benchmarks shows that simply increasing the inference budget of the base model yields only modest gains. Such tasks demand grounded perception, external knowledge, and numerical measurements obtained through heterogeneous tools, whose outputs must be made interpretable to the model and retained across reasoning steps. We introduce OpenVLHarness, a multimodal reasoning harness organized around a capability layer and persistent multimodal memory. The capability layer combines task-level tool interfaces with evidence rendering, providing consistent access to heterogeneous backends and converting their outputs into model-readable observations. Persistent multimodal memory retains the corresponding images, structured data, and web references as addressable artifacts throughout each query. A coding interface executes generated programs over stored data, enabling computations that combine results from multiple tools. Together, these mechanisms support an iterative process of acquiring, inspecting, and composing multimodal evidence. Evaluated on 22 benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by 9.7-12.6 points over matched no-tool baselines, including +11.2 with Qwen3-VL-8B, +9.7 with Kimi K3, and +9.9 with GPT-6 Astra. With GPT-6 Luna at medium reasoning effort, OpenVLHarness outperforms the base model, Codex, and Visual Sketchpad at maximum effort by 7.7-8.5 points while incurring at most 37% of their model-token cost.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新