AI审稿人盯着附录挑刺
让 AI 帮忙审论文,它一半以上的「弱点」是从附录里挖出来的——而人类审稿人只有 8.4% 的意见来自附录。研究者查了 ACL、EMNLP、COLM、ICLR 的 2000 篇论文,发现最新的 GPT 和 Claude 不管用 API、编程代理还是网页访问,都爱钻进附录深处,揪那些跟论文核心贡献无关的细枝末节;人则盯着正文的表达和核心想法。这不是你明天能用上的东西,但如果你投过稿、或以后要靠 AI 预审,它说明一件事:AI 挑出的「问题多」不等于「问题重要」。
📄 原文摘要(英文)
Large language models (LLMs) are increasingly used by reviewers to highlight weaknesses during the reviewing process, and their outputs often shape the reviews that humans submit. Prior work studies how LLM reviews align with human ones, yet we know little about which parts of a paper these models actually criticize. In this work, we investigate 2,000 papers from ACL, EMNLP, COLM, and ICLR, and show that the latest GPT and Claude models draw about 50% or more of their review weaknesses from the appendix. This behavior holds across the API, coding-agent, and web access methods for using LLMs to review a paper. In contrast, we find that humans draw only 8.4% of their weaknesses from the appendix, and these weaknesses primarily revolve around a paper's presentation and content related to its core idea. Meanwhile, LLMs draw weaknesses from deep in the appendix that concern fine-grained technical details not central to the paper's contributions. We anticipate that our analysis will contribute to the scholarly discourse on AI reviews. Our code is publicly available at: https://github.com/cincynlp/review_appendix.