AI Pulse
📄 论文解读

手机AI不用人教,自己学会用App

手机上的AI助手(比如能帮你点外卖、设闹钟的那种)通常需要大量人工标注才能学会操作新App,但App更新快、数量多,根本标不过来。这篇论文让AI自己探索App、自己生成任务、自己给自己打分,完全不用人工标注。核心是两招:一是让AI在真实App里乱点并记录结果,二是把“整件事做没做成”“每一步对不对”“错了怎么改”三层反馈拆开,分别用来优化AI的策略。最终,一个8B参数的模型在AndroidWorld测试中达到67.2%的成功率,接近需要人工数据的专业模型(69%);改进版甚至达到77.6%,成为开源最强。它不是你明天就能用的产品,但指明了方向:未来的手机AI助手可能不需要开发者专门适配,自己就能学会用任何App。

📄 原文摘要(英文)

MLLM-based mobile GUI agents have made substantial progress in UI understanding and action execution, but adapting them to real target apps remains costly because mobile apps are numerous, frequently updated, and hard to cover with human-written tasks, demonstrations, or reward labels. Existing annotation-free GUI learning reduces manual supervision, yet lacks a unified substrate connecting target-app exploration, curriculum mining, rollout execution, and feedback, while policy optimization often relies on isolated rollouts and coarse rewards that are hard to convert into reliable improvement signals. We present MobileForge, an annotation-free adaptation system for mobile GUI agents. MobileForge consists of MobileGym, which grounds task generation and rollout evaluation in real mobile app interaction, and Hierarchical Feedback-Guided Policy Optimization (HiFPO), which turns trajectory outcomes, step-level process feedback, and corrective hints into hint-contextualized step-level GRPO updates. Using only automatically generated annotation-free adaptation data, MobileForge adapts Qwen3-VL-8B to 67.2% Pass@3 on AndroidWorld, close to the closed-data GUI-specialized GUI-Owl-1.5-8B base model at 69.0%. The MobileForge-adapted ForgeOwl-8B further reaches 77.6% Pass@3 on AndroidWorld and 41.0% success on the out-of-domain MobileWorld GUI-only split, establishing the strongest open-data mobile GUI agent in our evaluation. Code, data, and trained models will be released at https://mobile-forge.github.io/.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新