AI 开始自己动手操作你的手机和电脑了
我们习惯让 AI 聊天、写代码、画图；这篇报告展示的 AI 直接接管你的设备：在手机上点开 App、在电脑上敲命令、在网页里填表，还能跨平台连续干几十步的活。它叫 Qwen-UI-Agent，来自通义团队，在手机操作测试里拿到 82% 到 97% 的完成率，电脑和浏览器任务上也追平甚至超过 GPT-5.6 这类顶级模型。它不只是按固定脚本走，而是像人一样看着屏幕、判断下一步、出错了自己改。它还能同时用鼠标点击和命令行，把两种操作混在一起。更关键的是，它用大量真实手机环境训练，不是只在模拟器里玩。这离「你喊一句，它帮你把事办完」的通用数字助理又近了一步——但别指望明天就能用，它目前还是实验室里的前沿能力。
📄 原文摘要(英文)
GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision agents that operate reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI execution, complete long-horizon tasks, proactively initiate useful services, and autonomously improve their capabilities with minimal human effort. Guided by this vision, we present Qwen-UI-Agent, a real-world centric foundation GUI agent spanning mobile, computer-use, web, and DeepSearch environments. Qwen-UI-Agent combines diverse sandbox environments with a large-scale real-device mobile runtime. Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn. An AutoResearch-style data flywheel uses agents to construct tasks and environments, diagnose failures, and plan subsequent iterations. Online RL supports training on trajectories exceeding 100 turns, with over 10,000 concurrent environments accelerating rollout. A lightweight harness layer supports proactive service initiation and stateful workflows across mobile and computer. Across a broad suite of evaluations, Qwen-UI-Agent sets state-of-the-art performance on mobile-use benchmarks while delivering competitive performance on computer- and browser-use tasks against frontier models, including Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol. On mobile use, it achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. On computer use, it achieves 79.5% on OSWorld-Verified and a 40.0% partial-progress score on OSWorld-v2. On browser use and GUI grounding, it achieves 73.6% on WebArena and 81.5% on ScreenSpot-Pro, respectively.