AI Pulse
📄 论文解读

一个模型接管手机、电脑和网页操作

现在的AI操作数字界面,基本是各管一摊:管手机的、管电脑的、管网页的,各练各的。这篇把三样活儿塞进同一个模型——看懂屏幕上的按钮、跨应用多步操作、直接调用视觉工具——2B到8B三个尺寸都练,结果在手机、桌面、网页各自的标准测试里,追平甚至超过了那些专攻单一领域的模型。它靠的不是更聪明的算法,而是一套能同时跑几百个真实环境的训练设施,让模型在真实操作中自己学。它不是你明天能用上的东西,但它是「一个AI替你操作所有设备」从口号变成工程现实的一步。

📄 原文摘要(英文)

We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities. To enable this, we develop a scalable environment and reinforcement learning (RL) infrastructure. On the environment side, we host hundreds of concurrent instances across heterogeneous per-domain backends, serving both trajectory data collection and online RL. To enable efficient and scalable RL training, an asynchronous framework keeps explicit control over the cross-domain training distribution and remains stable under noisy environment feedback and off-policy drift. Experimental results show that MintAct achieves state-of-the-art performance (48.9 on OSWorld-Verified) across a wide range of benchmarks at comparable model sizes.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新