AI做科研，瓶颈不在脑子，在跑实验
训练AI做科研，大家以为瓶颈是模型不够聪明，其实是跑实验太贵：AI每想出一个方案，都要在真实环境里跑一遍，独占一台机器、花真时间。这篇论文发现，生成方案可以批量并行、成本摊薄，但执行方案没法共享，于是训练成本被实验环节卡死。他们的解法是让AI先学一个「世界模型」——用模型模拟实验环境，省掉真实执行。但模拟有偏差和噪声，于是又加了两招：在线去偏和方差加权降噪，理论上证明收敛更快，实际训练提速3-4倍，效果还超过标准方法。更狠的是，他们用这个方法训出的4B和9B小模型，在留出基准上打赢了48B和120B的开源大模型。这不是你明天能用上的东西，但它指向一个趋势：AI做科研的竞争，正在从「谁模型大」转向「谁跑实验便宜」。
📄 原文摘要(英文)
Automating empirical research is a long-standing direction of AI. Recent automatic research (AutoResearch) agents bring this goal within reach, as modern LLMs show the capability to independently implement solutions and learn from the execution outcomes. Behind these gains, post-training (especially RL) plays a central role. In this paper, we identify a fundamental tension when scaling RL for these agents: the two components of every AutoResearch trajectory (agent generation and environment execution) scale in very different manners, since all generation shares compute through batching, while each execution occupies its exclusive sandbox and real machine time. As a result, the environment execution dominates the training cost and becomes the bottleneck as trajectories grow. To resolve this tension, we propose World Model RL (WMRL), which replaces environment execution with a world model to remove this bottleneck. Additionally, the world model can be imperfect, as its rewards are corrupted by bias and noise. Therefore, we further equip WMRL with two mitigations, Online Debiasing and Inverse-Variance Denoising, which offset the bias and suppress the noise respectively. Theoretically, we prove that both mitigations of WMRL strictly improve the convergence guarantee. Empirically, WMRL accelerates training by 3-4x on various tasks at different agent scales, while exceeding the performance of standard RL baselines. Moreover, our post-trained 4B and 9B agents outperform much larger open-weight agents of 48B and 120B on held-out benchmarks. Beyond AutoResearch, WMRL also transfers to post-training embodied VLA policies, which demonstrates the generalizability of our method.