AI 干活只看结果？支付场景里，过程违规照样赔钱
现在的 AI 智能体评测，基本只看最后任务成没成。但真实商业里，过程比结果更致命：在支付这种高敏感领域，你哪怕最终把钱转对了，中间一步踩了过时政策或冲突规则，罚款和损失已经发生了。BizBench 把一家头部支付公司的真实流程搬进测试环境，789 份知识文档、601 个业务工具，50 个任务里塞满资源限制、时间依赖、历史承诺和互相矛盾的旧指引。结果：最强模型 Claude Opus 5 严格成功率只有 64%,GPT-6 Sol 是 61.3%;更扎眼的是，一个开源模型目标达成率 42%，但严格成功率掉到 32.7%——也就是说，有近四分之一的活儿，它结果是好的，过程是违规的。还有个模型平均每次任务调用 191 次工具，比标准答案多 70%，纯粹在瞎折腾。它不是你明天能直接用的东西，但它给所有吹「AI 替你干活」的人提了个醒：结果对了，不等于没闯祸。
📄 原文摘要(英文)
Existing long-horizon agent benchmarks often emphasize final task success while providing limited assessment of intermediate execution validity. Yet real-world business operations require agents to navigate extensive, potentially outdated or conflicting policies while satisfying critical constraints throughout execution. These requirements are particularly consequential in high-stakes domains such as payments, where intermediate violations can incur financial losses or regulatory consequences regardless of the final outcome. To bridge this evaluation gap, we introduce BizBench, a multi-turn benchmark grounded in real-world procedures from a leading payments company. Its executable environment integrates 789 knowledge documents and 601 discoverable business tools, with consistent policies, states, and tool behavior. The benchmark comprises 50 tasks involving resource constraints, temporal dependencies, prior commitments, and conflicting or outdated guidance. Each task is validated through reference execution, while deterministic checks assess both goal attainment and execution validity without prescribing a single solution path. Experiments reveal substantial limitations even among frontier models: Claude Opus 5 and GPT-6 Sol achieve strict success rates of only 64.0% and 61.3%, respectively. For Qwen3.8-27B, goal attainment reaches 42.0%, but strict success drops to 32.7%, demonstrating how outcome-based evaluation can mask invalid execution. Beyond success rates, DeepSeek-V4.1-Flash averages 191.3 tool calls per run, approximately 70% more than reference executions, revealing substantial execution overhead. Overall, BizBench establishes a testbed for evaluating and advancing long-horizon agents beyond task completion, toward reliable and constraint-compliant execution in complex business environments.