给AI看图答案打分:不光看对错,拆开看理由
AI 看图片回答问题时,经常看起来头头是道,其实一句关键物件或数据是它瞎编的。现在的训法只判整段对错,它不知道错在哪一步,就没法改。这篇研究像老师批作文一样,把标准答案拆成一条条“事实”,再按“有没有看图”、“推理顺不顺”、“听没听指令”三方面分别打分,让 AI 精确知道哪步错了。在 8B 模型上比传统方法提升明显,尤其考验知识和看图推理的题。它不是你能明天就用上的产品,但这是让 AI 更少“一本正经胡说八道”的一个关键技术方向。
📄 原文摘要(英文)
Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.