AI Pulse
📄 论文解读

AI 代理的「肌肉记忆」:学一次,换模型也能用

大模型代理现在能像人一样积累「肌肉记忆」——不是每次从头学,而是把重复任务固化成可复用的技能。研究者造了一个涵盖 382 个真实企业任务的测试集,覆盖 6 种职业角色和 22 项技能,发现:只要一次优化,代理的整体表现就能提升 3.7-6.7 分;更关键的是,用多个不同模型生成的执行轨迹来训练技能,换到新模型时准确率仍达 73.1%,远超只用单一模型。但有些技能会「偏科」——在特定角色上越强,换到别的角色就越弱。这不是你明天能直接用的工具,但它告诉你:未来企业部署 AI 代理时,可以像培训员工一样,先让它在多个模型上「轮岗」,再固化技能,这样换模型也不怕。

📄 原文摘要(英文)

Procedural memory is increasingly used to improve LLM agents on recurring workplace tasks, yet its ability to produce reusable skills remains poorly understood. We introduce AFTER, a benchmark of 382 realistic enterprise tasks spanning six professional roles and 22 procedural skills, designed to evaluate how skills transfer across tasks, roles, and model backbones. The benchmark includes controlled evaluation settings for local improvement, cross-task transfer, cross-role transfer, and cross-model generalization. Experiments show that procedural memory delivers consistent gains in industrial workflows: a single refinement round improves aggregate performance by 3.7-6.7 points, while skills evolved from diverse multi-model execution traces achieve 73.1% cross-model test accuracy, outperforming all single-model trace sources. We further find that some skills generalize broadly across tasks and models, whereas others become specialized to role-specific workflows and lose effectiveness under transfer. These results provide practical guidance for building, evaluating, and deploying procedural memory systems in production agent platforms.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新