AI 的“技能”会过期，这篇逼它学会换工具
我们总以为 AI 学会一个工具就能一直用；这篇告诉你，工具生态本身在变，旧技能会失效，新工具会冒出来。研究者造了一个会“演化”的测试场：42 个领域、1588 个任务，工具会消失、被替换、出现更高效的替代品，文档还故意写得含糊。结果最强模型也只有 42.3% 成功率，而且现有的“技能学习”方法在生态变化时反而帮不上忙。他们因此提出一个叫 Steward 的管理器，边干活边维护技能、动态调配工具，在生态变动中维持了更高成功率。它不是你明天能用上的东西，但它点破了一个被忽视的问题：AI 的熟练，可能只是对某个静止世界的熟练。
📄 原文摘要(英文)
In this paper, we introduce Δτ-Bench, an executable agent benchmark that models the structural evolution of large tool ecosystems. In Δτ-Bench, independently evolving services continually reshape how tasks can be accomplished: familiar tools may disappear or be replaced, while more efficient alternatives emerge amid overlapping functionality and noisy documentation. Against this moving capability frontier, we evaluate whether agents can proactively discover efficient tool-use paths and promptly revise their accumulated procedural knowledge, bringing together demands that existing benchmarks fail to examine. Δτ-Bench comprises 1,588 validated multi-turn tasks across 42 domains and six successive ecosystem states generated through capability-preserving transformations. Our evaluation shows that even the strongest evaluated model or harness achieves only 42.3% task success, while initial gains from existing skill-learning methods can not consistently translate into better adaptation as the ecosystem evolves. These findings motivate Steward, an ecosystem manager that jointly manages tools and procedural knowledge through incremental skill maintenance and dynamic provisioning during task execution. Experiments show that Steward sustains higher task success across ecosystem changes with lower skill-maintenance overhead, while improving both recovery from disrupted workflows and adoption of more efficient alternatives.