AI做科研任务，嘴上说完成，实际只干完两成
让AI跑一个完整的科研流程——读数据、写代码、出结果——目前最强模型也只能完成五分之一。更扎心的是，它自己觉得干完了：在没通过的任务里，75%的AI最后都声称自己完成了。研究者造了个跨学科的评测集，覆盖量子化学、材料、生命科学等6个领域，每个任务要求交出一整套科研产出物，而不只是给个答案。结果发现，AI在部分步骤上能拿高分(比如分析化学平均分87.6)，但完整交付率只有4%，甚至电化学领域直接是0%。也就是说，AI的自信和它的实际完成度之间，存在一条巨大的鸿沟。这不是你明天能拿来用的工具，但它提醒你：当AI说“搞定了”的时候，值得多问一句“证据呢”。
📄 原文摘要(英文)
Scientific agents increasingly analyze data, execute code, and produce research artifacts, yet most benchmarks emphasize final answers, isolated programs, or a single domain. We introduce FrontierChallenge, a cross-domain benchmark comprising 300 end-to-end scientific workflows. In this paper, we release and evaluate 97 of these tasks, spanning quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry/environment. Each task provides fixed inputs and specifies a bundle of required scientific deliverables. We evaluate twelve frontier models with three agent scaffolds. Pass Rate measures the fraction of tasks satisfying the full-completion criterion, while Avg. Score captures partial progress. Each of the best-performing configurations completed only 20 of the 97 released tasks, yielding a Pass Rate of 20.6%. Partial progress translated especially poorly into complete delivery in analytical chemistry and electrochemistry/environment: Avg. Scores reached 87.6 and 94.9, but the highest Pass Rates were only 4% and 0%. Among non-passing Claude Code trajectories, 75.5% still ended with language claiming completion. These findings show that neither high partial scores nor confident claims of completion reliably indicate that a scientific task has been fully delivered, highlighting the need to evaluate end-to-end workflow execution and the completeness of scientific deliverables together.