AI Pulse
📄 论文解读

AI 写的审稿意见,越来越像同一个人

同行评审靠的是几个读者各自挑出不同的问题。这篇量了这件事:ICLR 上同一篇论文的评审意见重合度从 2026 年的 12.0% 涨到 2027 年的 21.3%,NeurIPS、ICML 和 F1000、PLOS 的期刊也逐年上升。第三位审稿人给一篇论文带来的新问题,2027 年是 5.46 个,2026 年是 6.12 个。他们用检测 AI 生成文本的工具 MELD 扫了所有评审意见,发现同一篇论文里被标记为 AI 写的两份意见,重合度比两份没被标记的高:ICLR 2024-2025 高 3.0 个百分点,2026-2027 高 4.5 个百分点,五家期刊高 5.4 个百分点。结论很直接:被 AI 标记的评审意见彼此更不像独立判断,一篇论文收到的批评正在趋同。这不是你明天能用上的工具,但它改变的是「论文值不值得信」这条链的源头。

📄 原文摘要(英文)

Peer review relies on several readers finding different problems. We measure critique overlap in conference and journal reviews. At ICLR, measured overlap rose from 12.0% in 2026 to 21.3% in 2027, in reviews from the same stage of the review process, with year-on-year rises at NeurIPS, ICML and the F1000 and PLOS journals. A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026. We apply MELD, a detector of LLM-generated text, to all reviews. Within a paper, two reviews that MELD flags overlap more than two unflagged reviews: by 3.0 points at ICLR in 2024-2025, by 4.5 points in 2026-2027, and by 5.4 points across five journal sources. These results provide evidence that LLM-flagged reviews of one paper are less distinct from each other than unflagged reviews, and suggest that the diversity of critiques among reviews of a single paper is falling.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新