AI学会自己修复被损坏的图片
你给AI看一张模糊、有噪点或被涂黑的照片,它通常就懵了。但这篇让AI自己先修复图片,再理解内容——就像人先擦擦眼镜再看东西。研究者用三步训练:先教它重建原图,再用奖励机制让修复更清晰、语义更准,最后把原图和修复图一起用于推理。在真实损坏测试中,效果超过所有现有方法。它不是你明天能用上的,但揭示了一个方向:让AI像人一样,先处理感官输入缺陷,再思考。
📄 原文摘要(英文)
Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly under real-world visual corruptions. While existing robustness enhancement approaches exist, they are limited: black-box feature alignment lacks interpretability, and white-box text-based reasoning cannot restore lost pixel-level details. This work investigates a fundamental research question: Can MLLMs recover corrupted visual content by themselves? To address this, we propose Robust-U1, a novel framework that equips MLLMs with explicit visual self-recovery capability for robust understanding. The approach comprises three core stages: supervised fine-tuning for initial reconstruction, reinforcement learning with dual rewards (pixel-level SSIM and semantic-level CLIP similarity) for aligning high visual quality, and multimodal reasoning that jointly considers both the corrupted input and the recovered image. Extensive experiments demonstrate that Robust-U1 achieves state-of-the-art robustness on the real-world corruption benchmark and maintains superior performance under adversarial corruptions on general VQA benchmarks. Analysis confirms that high-quality visual recovery directly enhances reasoning performance, establishing self-recovery as a critical mechanism for robust visual understanding. The source code is available at https://github.com/jqtangust/Robust-U1.