AI Pulse
📄 论文解读

AI 多智能体一遇冲突就翻车:最强模型也栽在锁和交接上

我们总以为 AI 协作是各司其职、信息一致;这篇告诉你,现实里 AI 之间也会互相打架——一个说文件已更新,另一个还在用旧版本,或者两个都抢着改同一个文件。研究者造了个叫 CLASHarness 的测试场,能自动生成 600 个带冲突的多轮任务,让 AI 在电脑上像人一样协作完成。结果发现三个硬伤:模型容易盲信同伴的错话而不去核对屏幕;发现冲突容易、修复冲突难;最要命的是收尾动作——超过 58% 的关闭操作失败,模型经常忘了释放共享锁或交接任务。这不是你明天能用上的东西,但它划出了一条线:多智能体系统要真落地,先得解决“AI 之间怎么处理分歧”这个基础问题。

📄 原文摘要(英文)

Vision-language models (VLMs) are increasingly deployed in multi-agent systems to automate complex computer-use workflows, yet existing benchmarks mostly assume consistent, cooperative environments. In contrast, real-world collaboration rarely runs on consistent information: peer or sub-agents frequently report outdated states, issue contradictory instructions, or compete over shared permissions and files, forcing agents to check others' outputs and verify ground truth in the environment. Evaluating such conflict resolution remains challenging: manual authoring cannot scale across complex systems, while unconstrained LLM generation produces narrative drift, answer leakage, and unsatisfiable rubrics. We introduce CLASHarness (Conflict-Loaded Agent Scenario Harness), a controllable framework for generating verifiable multi-agent conflicts in computer use, with a benchmark of 600 multi-round tasks. By combining hierarchical scenario planning with an eight-step verified micro-pipeline, CLASHarness pre-commits decisive visual facts before rendering and compiles hybrid code and semantic graders directly from executed reference trajectories, guaranteeing every task is solvable by construction. Evaluating 15 frontier models on CLASHarness reveals three core bottlenecks. First, when sub-agent outputs conflict with the environment, weaker models blindly follow misleading information, whereas stronger models inspect visual evidence to verify ground truth. Second, resolving conflicts is much harder than finding them: while agents can spot inconsistencies, actively fixing system state drops task success significantly. Third, coordination protocols remain a universal bottleneck: even frontier models fail over 58% of closing actions, often forgetting to release shared locks or hand off tasks. CLASHarness provides a scalable, verifiable foundation for building reliable multi-agent systems.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新