AI 改代码，十家模型没一家能完整干完活
让 AI 按你的话改一个真实 App，它可能改对七成，但能从头到尾干完的不到三成。研究者从 2.6 万次真实用户编码会话里筛出 273 个任务，让十个主流模型去改，最强的一个平均能完成每个任务 69.9% 的要求，但只有 26.4% 的任务被它完整搞定。更扎心的是，改动越复杂——比如同时改三四处地方——所有模型的表现都明显下滑。这不是某个模型的锅，是当前 AI 改代码能力的普遍天花板：它擅长局部修补，但撑不起一次完整的、多步骤的交付。
📄 原文摘要(英文)
Evaluating human-AI co-design requires testing whether coding agents can translate real users' multimodal requests into working application revisions. We introduce Co-Design Bench, a reference-free benchmark of 273 tasks curated from 37,463 deduplicated queries across 26,314 real user coding sessions. Its two tracks, Multimodal Editing (ME) and Repository Editing (RE), cover localized edits with optional image, audio, video, or document inputs and coordinated repository revisions. Tasks span three capability dimensions: Grounded Understanding, UI Transformation, and Functional Implementation. The tasks pair normalized requests and runnable pre-edit applications with 2,331 hidden expert-authored acceptance criteria. Our judge-then-refute protocol grades criteria using source and browser evidence, challenges positive verdicts with a separate LLM verifier, and audits visual and interaction claims without gold code or target screenshots. Under a shared agentic harness, the strongest of ten model endpoints passes an average of 69.9% of criteria per task but fully satisfies only 26.4% of tasks. All ten endpoints achieve lower criterion completion on repository edits spanning three or four change types than on those spanning one or two. Co-Design Bench provides a testbed for studying the gap between partial requirement satisfaction and complete task success under agentic execution for real-user application revisions.