AI 干长活失败时，它自己都说不清哪步错的
AI 干长活(比如自动订票、跑多步流程)失败时，现在的评测只告诉你「结果错了」，不告诉你「哪一步开始错的、该怪谁」。这篇搞了个新测试集：1,140 条真实失败轨迹，平均 145 步，让人类标注出「责任角色」和「最早的关键错误步骤」。结果最强基线只能猜对 13.2% 的错误步骤——说明长流程的故障定位远没解决。作者顺手给了一个不用训练的方法，靠分段摘要回溯指令，把责任角色猜对率提到 51.1%，但精确到步骤还是只有 24.1%。它不是你明天能用上的工具，但给了一个信号：AI 干活越久，出错后越需要「事故复盘」，而这块现在几乎是空白。
📄 原文摘要(英文)
When a long-horizon agent execution fails, outcome-level evaluation reveals the unsuccessful result but not where the decisive error entered the trajectory. Developers must then inspect the full execution to identify the responsible role and localize the earliest decisive root-cause step. Existing failure-attribution benchmarks largely focus on shorter traces, leaving diagnosis across hundreds of recorded steps underexplored. We introduce LongRCA Bench, comprising 1,140 failed trajectories across five domains without injected errors. It provides independently scored human labels for the responsible role and earliest decisive root-cause step. The median trajectory contains 145 steps, and the strongest baseline reaches only 13.2% exact root-step accuracy. We further present Root-Cause Trajectory Attribution (RCTA), a training-free method that retrieves candidate error steps from segment summaries and traces them to available earlier handoff instructions. Using the same backbone, benchmark instances, and scoring protocol, RCTA reaches 51.1% responsible-role accuracy and 24.1% exact root-step accuracy. These results highlight the need to evaluate responsible-role attribution and exact root-step localization as separate targets in long-trajectory failure diagnosis.