AI编程助手找bug像大海捞针？新测试揭底
现在的AI编程助手在修bug时，往往被当成一个黑箱：给个任务，看它能不能修好。但修不好时，你根本不知道它是没找到问题代码，还是找到了但改错了。这篇论文拆开了这个黑箱，专门测试AI的「探索能力」——给定一个代码仓库和一个bug报告，AI需要在一个固定的代码行数预算内，列出最可能相关的代码区域，并按重要性排序。研究者从848个真实bug中提取了人类和AI成功修复时实际查看的代码行作为标准答案。结果发现：当前最好的AI探索者已经能准确找到文件，但精确到具体哪一行、以及如何高效排序，仍然是顶尖模型之间的分水岭。换句话说，你的AI助手可能知道问题在哪个文件，但具体是哪几行代码，它还在瞎猜。这不是你明天就能用的工具，但它让你看清：AI编程助手离真正理解代码库还有多远。
📄 原文摘要(英文)
Repository-level coding benchmarks such as SWE-bench have driven a rapid surge in the capabilities of coding agents. Yet they usually treat coding tasks as a holistic, binary prediction problem (e.g., resolved or unresolved), neglecting fine-grained agent capabilities such as repository understanding, context retrieval, code localization, and bug diagnosis. In this paper, we introduce SWE-Explore, a benchmark that isolates the evaluation of repository exploration, a critical capability of coding agents. Given a repository and an issue, SWE-Explore asks an explorer to return a ranked list of relevant code regions under a fixed line budget. SWE-Explore covers 848 issues across 10 programming languages and 203 open-source repositories. For each instance, we derive line-level ground truth from independent agent trajectories that successfully solved the same issue, distilling the specific code regions their solution paths actually consulted. We evaluate exploration along coverage, ranking, and context-efficiency dimensions, showing that these metrics strongly track downstream repair behavior. Across a broad set of retrieval methods, general coding agents, and specialized localizers, we find that agentic explorers form a clear tier above classical retrieval. While file-level localization is already strong for modern methods, line-level coverage and efficient ranking remain the key axes differentiating state-of-the-art explorers.