AI找论文，输给科学家的直觉
科学家最厉害的能力之一，是从海量论文里嗅出「这个问题需要那篇老文章」。这篇研究让184位计算机论文作者亲手标注：哪些早期论文真正推动了自己的项目，然后考AI——给你一个研究问题，只能看当时已有的文献，能不能把这些关键论文找出来。结果：最强的智能体检索，命中率只有0.51，和普通向量检索的0.48几乎没差。也就是说，AI能读论文、能写论文，但「哪篇旧文能点亮一个新想法」这种跨领域的直觉，它还没学会。这不是你明天能用上的工具，但它划出了一条分界线：科学发现里最像「品味」的那部分，目前还是人类的领地。
📄 原文摘要(英文)
What makes great scientists great? Even as AI systems start to make progress on open problems, scientists remain far ahead of them at sensing which prior idea, buried in an ever-growing archive of research, a new problem needs. To study this skill, we draw on researchers who know firsthand which earlier work advanced their completed projects, with papers serving as pointers to the ideas within. Using our automated pipeline that makes author annotation scalable, we build ScholarCatalyst by having 184 lead authors of 207 recent computer science papers label which candidates did or could have advanced their project, each with a detailed rationale. We introduce a retrieval task with author-provided judgments: given an initial research question, retrieve these papers from only the literature available when the project began. Agentic search does no better than embedding retrieval (0.42 vs. 0.48 Recall@20) despite calling that same retriever as a tool. Even an agent built on Claude Fable 5.1, which may have seen the completed papers during training, reaches only 0.51 R@20. These results highlight the need for new training recipes that equip models with expert intuition for searching broad corpora. We envision ScholarCatalyst as a step toward scientific agents that can take a half-formed idea and point to the prior research it needs.