AI Pulse
📄 论文解读

AI离自主科研还差得远:撤掉指导,能力暴跌

我们总以为AI已经能自己搞科研了。这篇论文用60个真实研究项目测了18种最强AI配置,结果很扎心:只要把人类的方法指导撤掉,AI的得分直接从50.91掉到26.62,几乎腰斩。换句话说,现在的AI更像一个听话的实习生,你告诉它每一步怎么做,它能干得不错;一旦让它自己选方法、自己定路线,它就抓瞎了。这不是某个模型的短板,而是整个行业的现状。它不是你明天能用上的东西,但它划出了一条诚实的线:我们离“AI科学家”还很远,别被演示视频骗了。

📄 原文摘要(英文)

Artificial superintelligence (ASI) requires AI to move beyond mastering existing knowledge toward exploring the unknown, creating new knowledge, and turning new ideas into verifiable results. However, the capabilities of today's AI systems are still largely built on learning, compressing, and applying existing human knowledge. Accordingly, existing benchmarks primarily test whether AI can produce correct answers based on learned knowledge, or whether it can complete tasks under extensive human guidance. We therefore introduce ASI-Bench, the first benchmark to jointly evaluate AI systems' capabilities of innovative exploration and autonomous scientific execution across general research domains, and the first to progressively withdraw human methodological guidance within the same research project to test how far AI can proceed on its own. Built by over 40 experts with the cost of 31,000+ human hours, ASI-Bench contains 60 project-level research tasks across 11 scientific domains and progressively reduces methodological guidance to test whether AI can independently select methods, conduct research, and produce verifiable results. All tasks undergo expert review, AI-assisted auditing, sandbox execution, and scorer validation. Across 18 state-of-the-art agent--model configurations, the average score drops from 50.91 with full methodological guidance to 29.10 with only the method specified and 26.62 when agents must determine the method themselves. This sharp decline shows that current systems remain heavily dependent on human guidance and are still far from autonomously conducting end-to-end, project-level scientific research. ASI-Bench is open to the world. We invite researchers and builders everywhere to contribute new tasks, challenge the limits of today's AI, and help accelerate humanity's collective path toward artificial superintelligence at https://asibench.apexin.ai/submit.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新