AI 训练环境不再死板,能自动找弱点
AI 智能体在静态环境里训练,就像在固定跑道练跑步——跑熟了就没进步。这篇提出 EnvHarness,一个可编程的插件层,能动态改造现有环境,不碰底层逻辑。配套的 EnvRigger 像教练,观察智能体执行轨迹,自动找出它的弱点,然后合成插件来针对性强化,再用新跑分验证。在四个领域的五个基准上,EnvHarness 比原始环境和专用生成方法都好,最高提升 9 个点,还少用 9.8% 的执行步数。它不是你明天能用上的,但指向一个趋势:AI 训练环境从静态走向自适应,智能体与环境能持续共同进化。
📄 原文摘要(英文)
LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.