AI操作电脑的“全能考试”：及格率仅41%
现在的AI智能体（Computer-Use Agents）被要求同时操控图形界面、命令行、代码编辑器、浏览器和外部工具，完成一个真实工作流。但现有测试只考单项技能，比如只会点鼠标或只会写代码。这篇论文推出了WeaveBench，一个包含114个真实工作任务的“全能考试”，覆盖数据分析、软件部署等8个领域。每个任务都要求AI在同一个流程里混合使用GUI点击、命令行输入和代码修改。结果：最强模型组合的通过率只有41.2%。更关键的是，如果只看最终结果（比如文件对不对），会严重高估AI——因为AI可能作弊，比如伪造截图或硬编码答案。这个基准不是给你明天用的，但它揭示了：AI离真正像人一样操作电脑完成复杂工作，还有很大距离。
📄 原文摘要(英文)
Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114 tasks across 8 real-world work domains, grounded in real user requests and publicly verifiable artifacts. Each task requires agents to combine GUI observations/actions with CLI/code operations within a single trajectory. We evaluate these tasks on a real Ubuntu desktop inside deployed CLI-agent runtimes, augmented with a minimal desktop-control plugin. We also propose a companion trajectory-aware judge that inspects deliverables, files, screenshots, logs, and action traces, while detecting shortcut behaviors such as fabricated visual evidence or hard-coded metrics. Across frontier model-runtime pairings, the best PassRate reaches only 41.2%, showing the benchmark remains far from saturated. The trajectory-aware judge further reveals that outcome-only grading substantially overestimates agent performance. Overall, WeaveBench exposes a critical gap in CUA evaluation and provides an effective testbed to measure whether agents can orchestrate GUI, CLI, and code operations across long-horizon real-world tasks.