AI Pulse
📄 论文解读

AI学会自己修复被损坏的图片

你拍了一张照片,但光线太暗、有噪点或模糊,AI 就认不出来了。现在有个办法:让 AI 自己先修复图片,再理解它。研究者给多模态大模型加了三步训练:先学修复,再用奖励机制让修复更清晰、语义更准,最后同时看原图和修复图来推理。结果在真实损坏测试中达到最优,而且修复质量越高,推理越准。它不是你明天能用上的,但说明了一个方向:与其让 AI 硬扛坏图片,不如让它先自己修好。

📄 原文摘要(英文)

Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly under real-world visual corruptions. While existing robustness enhancement approaches exist, they are limited: black-box feature alignment lacks interpretability, and white-box text-based reasoning cannot restore lost pixel-level details. This work investigates a fundamental research question: Can MLLMs recover corrupted visual content by themselves? To address this, we propose Robust-U1, a novel framework that equips MLLMs with explicit visual self-recovery capability for robust understanding. The approach comprises three core stages: supervised fine-tuning for initial reconstruction, reinforcement learning with dual rewards (pixel-level SSIM and semantic-level CLIP similarity) for aligning high visual quality, and multimodal reasoning that jointly considers both the corrupted input and the recovered image. Extensive experiments demonstrate that Robust-U1 achieves state-of-the-art robustness on the real-world corruption benchmark and maintains superior performance under adversarial corruptions on general VQA benchmarks. Analysis confirms that high-quality visual recovery directly enhances reasoning performance, establishing self-recovery as a critical mechanism for robust visual understanding. The source code is available at https://github.com/jqtangust/Robust-U1.

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部