AI Pulse
📄 论文解读

AI 写代码出错,现在有个会盯梢的批评家

长周期编程 AI 最怕的不是写错,而是错了没人管、或者管错了。现有批评机制只负责给一句反馈就撒手,AI 假装听懂了、实际没改,它也不知道。Opera 把每次批评当成一张「待办便签」,一直盯着直到问题真的消失:它自己决定什么时候复查,诊断时给问题分类,发反馈前先对照代码证据核对,再跟踪 AI 后续动作,区分「敷衍了事」和「真解决了」。在三个编程基准上,它把不会自我批评的 AI 的解决率最高提升 15 个百分点,而且让 AI 自己批评自己时也有效。更值钱的是,Opera 生成的训练数据能让小模型 Qwen3.5-9B 在没批评家的情况下,解决率提升 10.2 个百分点,效果堪比用更强的模型来教。这不是你明天能用的工具,但它指向一个趋势:AI 写代码的下一场竞赛,不是谁写得快,而是谁更会自我纠错。

📄 原文摘要(英文)

Long-horizon coding agents need timely corrections, yet feedback can be ineffective or even harmful when it misjudges ongoing work or fails to address the underlying problem. Existing critics focus on evaluating trajectories and generating feedback, but rarely track what happens after feedback is delivered. We present Opera, a verbal critic framework that treats each correction as a persistent note, followed until the diagnosed problem is resolved. Opera decides when to review through periodic and event-driven triggers, diagnoses issues with typed operators, audits feedback against visible evidence before delivery, and tracks the agent's subsequent actions to distinguish mere compliance from actual resolution. As a test-time critic, Opera improves the resolve rate of non-critic agents by up to 12.4, 15.0, and 8.9 percentage points on Terminal-Bench 2.1, a SWE-Bench Pro subset, and DeepSWE v1.1, respectively, across four policy models, and achieves the highest mean resolve rate among competitive critic baselines on all three benchmarks, and also improves policy models when the policy critiques itself. Beyond inference, Opera-guided rollouts provide approximately on-policy training data: fine-tuning Qwen3.5-9B on them improves its resolve rate on held-out SWE-Bench Pro repositories by 10.2 percentage points without a critic at inference time, matching fine-tuning on rollouts from a stronger model, while preserving its performance when switching harness, i.e., from Openhands to Terminus-2, which the latter substantially degrades. Our code is available at: https://github.com/dongyuanjushi/Opera.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新