AI 看视频时“脑补”错了，现在有人教它分清“看到的”和“猜的”
多模态大模型看视频时，经常把画面里没有的东西说得像真的一样——比如画面里没人说话，它却“听”到一句台词。这不是它笨，而是它把“画面给的证据”和“自己脑补的常识”搅在了一起。这篇论文的做法很直接：先找出哪路信息在打架，再把模型内部混在一起的信号拆成“证据”和“猜测”两股，最后压住猜测、放大证据再输出答案。在 Qwen2.5-Omni 上，幻觉评测分数直接涨了 12.9 分，换到另一个模型也照样有效。它不是你明天能用上的工具，但它是“让 AI 别睁眼说瞎话”这条路上一个靠谱的方向。
📄 原文摘要(英文)
Recent Omni LLMs (OLLMs) have demonstrated remarkable capabilities in joint audio-visual reasoning, yet cross-modal hallucinations remain a major challenge. Existing mitigation methods mainly operate at the output level, overlooking the entanglement of grounding evidence and learned priors within fused representations. To address this issue, we propose Omni-Woodpecker (OWP), a training-free framework that shifts hallucination mitigation from output-level manipulation to conflict-conditioned representation rebalancing. Specifically, OWP first identifies cross-modal conflicts from complementary modality-specific views. It then employs orthogonal decomposition to separate grounding evidence from competing priors within the hidden states. Finally, OWP adaptively rebalances fused representations by suppressing misleading priors and restoring ignored evidence before decoding. Extensive experiments demonstrate the effectiveness and strong cross-architecture generalization of OWP. Using Qwen2.5-Omni-7B as the backbone, OWP improves performance by 12.9 and 8.6 points on CMM and AVHBench, respectively, while consistently achieving gains of 0.67-2.25 points across four general OLLM reasoning benchmarks. Furthermore, OWP generalizes effectively to VideoLLaMA2-AV, improving performance by 4.6 and 3.9 points on the same benchmarks. Code is publicly available at https://github.com/ZiiJianLiu/Omni-Woodpecker