AI 能选对方向，却做不出具体方案
让 AI 当科学家，它最缺的不是眼光，是手艺。研究者把 2025 年的文献快照喂给 6 个大模型，让它们写研究提案，再拿这些提案跟人类在 2026 年实际发表的论文比对：AI 在「引用哪些文献、关注哪些问题」上能到人类水平的 26%，在「具体想法」上只有 18%。换句话说，AI 知道该往哪走，却不知道该怎么迈步——它选的方向和人类社区一致，但落到具体方法、研究计划时就掉队了。更扎心的是，给 AI 更多领域背景、或者让更多 AI 一起想，都补不上这个差距。专家盲评也印证了这点：AI 提案输得最惨的不是表达，是「预期成功率」。这不是说 AI 提的是坏点子，而是它提的更像「没人会去做的方向」，而不是「被忽略的好机会」。它不是你明天能用上的工具，但如果你关心 AI 能不能真的做科研，这是目前最清楚的答案：瓶颈不在找方向，在把方向变成能落地的方案。
📄 原文摘要(英文)
Research taste, the judgment of what to read, which problems to pursue, and which methods to try, increasingly shapes the work of AI scientists, yet it remains far less understood than their ability to execute research. We introduce IdeaTaste, a benchmark that scores an LLM by how closely the research proposals it writes from a literature snapshot frozen at 2025 match the work the same communities published in 2026. Our experiments span six contemporary language models and four research areas from established to emerging fields, covering more than 90{,}000 proposals in total. LLM agents fall well short of human communities in both the literature they cite and the ideas they state, reaching about 26\% of the human level on citation matching and only 18\% on idea matching. Their populations can cite as diversely as a community and take up its tasks and methods, yet rarely arrive at the concrete approaches and research plans it pursues. Neither richer field context nor larger populations substantially close this gap. In blinded expert review, AI proposals trail the 2026 papers least in presentation and most in expected success, echoing their gap in concrete approaches, and those whose ideas no human paper shares fare worst, suggesting weaker directions rather than overlooked opportunities. These results indicate that the main bottleneck in AI research taste lies less in finding the directions a community works on than in turning them into the concrete approaches it goes on to pursue. We release IdeaTaste as an open, continuously updated leaderboard (https://cxcscmu.github.io/IdeaTaste) that benchmarks contemporary AI proposals against future publication cycles.