AI 自评成绩单，现在要查原始凭证了
AI 干完活，常常自己给自己打分，而且它可能根本没看全证据——环境里散落的状态、执行过程的记录，它可能漏掉，甚至直接采信自己嘴里说的『我成功了』。这篇提出一个叫 Sauron 的验证器，把打分这件事改成『先查证据再下结论』:它同时翻环境状态和执行轨迹，一个结论必须找到原始出处才认，还限制查询次数、把相关标准一起查来省算力，每张成绩单都附可审计的证据清单。在 100 个个人助理任务、2586 条人工标注标准上，它用 8 种模型做底座都拿到最高 F1，比最强基线 Gandalf 便宜 23%，在故意注入的 13 类故障(包括模拟奖励黑客)里能抓出 97% 的失败、只误报 2% 的正常情况。它不是你明天能用上的东西，但它指向一个关键转向：当 AI 开始给自己发奖励，『证据链』比『分数』更值得信任。
📄 原文摘要(英文)
Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across the environment state and the execution trace. A judge may overlook this evidence or accept unsupported claims, including an agent's own reports of success. When rubric verdicts serve as reinforcement-learning rewards, such errors can reinforce undesirable behavior. We introduce Sauron, an evidence-first agentic verifier that investigates both state and trace and relies on a claim only after independently validating it against its source. Bounded queries and joint investigation of related criteria reduce context use and redundant work, and each verdict carries an auditable receipt of its supporting evidence. Across 2,586 human-labeled criteria from 100 personal-assistant episodes, Sauron achieves the highest F1 of five verifiers on all eight backbones tested while costing less than Gandalf, the strongest baseline. With GPT-5.6 Sol, it reaches 0.96 F1 versus Gandalf's 0.92 at 23% lower cost, with a wider margin on process criteria. On a paired fault-injection suite covering 13 families, several of which simulate reward hacking, Sauron detects 97% of injected failures while flagging only 2% of clean controls, outperforming baselines even when they are given the full trace. On TRAIL, a benchmark for detecting and localizing real agent errors, Sauron leads both Gandalf and TRAIL's own judge on all three quality metrics.