AI 研究员的成果，现在要过「可复现」考试
AI 研究代理声称自己发现了新东西，但分数高不等于真发现——它可能只是记住了训练数据里的答案。这篇提出一套「发现认证协议」:把研究结果变成可执行的复现测试，让另一个拿到同样初始信息和公开资料的代理，在不看原代理操作过程的情况下，独立跑一遍，看能不能达到同样的数值目标。如果它真能复现，原结果就通过；如果复现不了，就触发否决。在 SQLite 优化和虚拟催化剂控制两个任务上，96 轮测试零复现，上限 0.0468。它不是你明天能用上的工具，但它给「AI 到底有没有真本事」提供了一个比分数更硬的证据标准。
📄 原文摘要(英文)
AI research agents combine prior knowledge, public sources, and experimental feedback to produce useful results. The Discovery Certification Protocol (DCP) turns claims about these results into executable recovery and feedback tests. Gate 1 validates useful improvement on sealed evaluation. Gate 2 gives matched agents the registered starting information and observed Web content while withholding the target research history. Every valid method reaching the numerical target supplies a recovery witness and triggers the Core veto. DCP Core requires adequate controls, zero observed recoveries, and a finite-sample bound on recovery in one fresh registered episode. Optional Gate 3 measures the average effect of truthful feedback relative to a specified neutral policy from a shared checkpoint. DCP Evidence adds this effect after independent null calibration and a registered effect margin. Two controlled audits exercise the complete protocol in SQLite optimization and virtual catalyst control under different models. Each produced zero recoveries in 96 episodes, with an upper bound of 0.0468. Each paired study yielded 30 truthful recoveries and zero neutral recoveries, with passing 60-pair null studies. Additional cases exercise Core, recovered, and audit-incomplete decisions. A deterministic, LLM-free verifier reproduces the decisions from frozen evidence. DCP provides a common evidence language for useful outcomes, alternative routes, and feedback effects across AI research.