AI智能体在动态环境中表现不佳,新方法提升适应性
大多数AI智能体测试都在静态环境中进行,但现实世界是动态变化的。新研究EvoArena模拟了终端、软件和社交领域的渐进式变化,发现当前智能体平均准确率仅39.6%。研究者提出EvoMem记忆范式,让智能体像打补丁一样记录环境变化历史,从而更好地适应动态任务。在EvoArena上平均提升1.5%,在标准测试GAIA和LoCoMo上分别提升6.1%和4.8%。这不是你明天就能用的技术,但它揭示了AI在真实世界中可靠部署的关键挑战:环境变化时,智能体需要学会更新自己的知识库。
📄 原文摘要(英文)
Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions. To address this gap, we introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. We further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory. Experiments show that current agents struggle on EvoArena, achieving an average accuracy of 39.6% across evolving terminal, software, and social-preference domains. EvoMem consistently improves performance, yielding an average gain of 1.5% on EvoArena and also improving standard benchmarks such as GAIA and LoCoMo by 6.1% and 4.8%. Beyond individual tasks, EvoMem further improves chain-level accuracy by 3.7% on EvoArena, where success requires completing a consecutive sequence of related evolutionary subtasks. Mechanistic analysis shows that EvoMem improves evidence capture in the memory, indicating better preservation of complete evolving environment states. Our results highlight the importance of modeling evolution in both evaluation and memory for reliable agent deployment.