AI看得到却数不清，缺的不是视力是“指认”
多模态大模型能看见画面里的所有东西，但一遇到“数一数”这种多步推理就乱套：它看到了全部，却记不清自己数到哪了。研究者把这种失败归因于一个此前被忽视的环节——“指认缺口”:模型没法把语言和画面里具体的物体、区域绑定起来。他们做了个巧妙的实验：场景、问题、答案全都不变，只改变“指认方式”(比如给物体编号)，结果匹配任务的指认方式让准确率最高提升25%，而乱指一通不仅没用、有时还更糟。这证明瓶颈不在“看”而在“指”，而且不用重新训练模型，光靠外部编号就能补上。它不是你明天能用上的功能，但它把AI翻车的原因从“眼神不好”修正成了“手没指到”，这决定了下一代多模态模型该往哪个方向使劲。
📄 原文摘要(英文)
Multimodal large language models can recognize visual content yet struggle to reason reliably over it across multiple steps. In counting, a model may see all objects but lose track of which ones it has counted. We argue that such failures, commonly attributed to perception, reveal a reference gap: the failure to bind language to the relevant visual entities, regions, or locations. Motivated by the theory of visual routines, we test whether external references such as numbering can narrow this gap. We introduce ReGap-Bench, a paired benchmark of five visual reasoning tasks in which the scene, question, and answer are fixed and only the reference varies. Results show that task-matched references improve accuracy by up to 25\%, while mismatched references provide little benefit or even hurt performance. These findings identify the reference gap as a bottleneck distinct from visual perception and show that constructing task-appropriate visual references can serve as a training-free visual skill for improving multimodal reasoning.