AI训练框架瘦身:让研究者自己就能改算法
主流强化学习框架像俄罗斯套娃——改一行算法要翻遍训练器、分布式后端、数据生成三层代码,研究者改一次成本极高。Molt 把整个框架压缩到一个人能记住的规模,AI 助手也能完整读通,改算法只需改一个普通程序。它用单异步循环同时处理多模态和混合专家模型,且保证模型只在自己生成的 token 上训练,不污染数据。性能不输英伟达的 Megatron 方案。这不是你明天能用的工具,但如果你好奇 AI 研究为什么慢,这就是答案:不是算法难,是框架太重。
📄 原文摘要(英文)
Agentic reinforcement learning research is constant algorithm modification, new estimators, new pipeline stages, new rollout schemes, and in mainstream frameworks each change threads through layers of trainer, distributed backend, and rollout glue: the cost lands on the researcher at every iteration. Molt is a PyTorch-native training framework built to keep that cost small: a codebase compact and clean enough for a researcher to hold in their head, and for an AI coding assistant to read and reason about in its entirety, so the algorithm flow can be traced and changed end to end. The agent is an ordinary program, and one asynchronous loop trains multimodal and mixture-of-experts policies while never training on a token it did not generate, consistent in tokens, policy versions, and model semantics. Leanness does not cost performance: under a matched, fully asynchronous protocol, Molt is statistically comparable to a state-of-the-art Megatron-based stack. Molt is open source and provides recipes and containers at https://github.com/NVIDIA-NeMo/labs-molt.