AI Pulse
📄 论文解读

水下修图评分,没一个懂人眼

水下照片没有干净原图可对照,所以修图效果一直靠自动打分来评判。这篇把 73 种打分方法拿去和人的真实偏好比:在五组人类评分数据里,让打分器在同一场景的不同版本中挑出「人也会挑的那张」,结果没有一个稳定对得上。最稳的 Q-Align 一致度只有 0.23,而两个普通人之间随便就有 0.41。常用的 UCIQE 更离谱,它偏爱的版本常被人否掉。研究者还做了两件更狠的事:对图片做简单编辑甚至加肉眼难辨的扰动,分数就能超过人真正喜欢的修图结果;让生成模型反复「微调」本该不变的照片,噪声涨了 2.5 倍,73 个分数里只有 22 个每一步都往下掉。它不是你明天能用上的工具,但下次看到「AI 修图效果提升 30%」,你可以先问一句:这 30% 是谁打的。

📄 原文摘要(英文)

Underwater image enhancement is challenging to evaluate due to the lack of clean reference images, so researchers rely on no-reference image quality assessment (NR-IQA) metrics. However, it remains unclear whether these metrics agree with human judgments. We evaluate 73 handcrafted underwater, learned underwater, vision-language and general-purpose metrics against human preferences on five references: four published opinion-score sets and a new paired-comparison study. For each metric, we measure how often it selects the same image as people when comparing versions of the same scene. We further test whether simple image edits and small adversarial perturbations raise metric scores above the enhancement people preferred, and whether scores decrease as repeated generative edits add noise. Our results show that no metric consistently matches human preferences. The best worst-case Cohen's kappa over the five references is only 0.23 (Q-Align), while two raters in our study agree with a median kappa of 0.41. The widely used UCIQE often favors edits that people reject. On a Nano Banana Pro replica chain that should leave the image unchanged, only 22 of 73 scores decrease at every checkpoint, while noise grows by 2.5 times. We also introduce TrustUIQA, a benchmark that combines the human references, our study choices, the edited, attacked and generated images, and the scores of every metric on every image. The code is available at https://github.com/TrustUIQA/TrustUIQA and TrustUIQA at https://huggingface.co/datasets/TrustUIQA/TrustUIQA.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新