AI Pulse
📄 论文解读

AI 能自己照着软件抄一个出来吗

现在的 AI 助手分两拨:一拨会点鼠标、看屏幕,另一拨会写代码。真实工作两样都得会,还得来回切换。这篇让 AI 对着一个正在运行的软件,自己决定什么时候看界面、什么时候写代码、什么时候跑起来验证,最终做出一个功能一样的复刻版。测试里最强的 GPT-6 Astra 综合得分 58.1%,但能通过全部程序化检查的任务只有 2.8%——界面长得像容易,交互和计算逻辑才是真坎。它不是你明天能用的东西,但这是 AI 从「会聊天」走向「真干活」的一个路标。

📄 原文摘要(英文)

Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Real digital work requires both, interleaved rather than stacked end to end. We study hybrid CUAs that autonomously decide when to explore an interface, implement software, and run and visually verify their artifacts. We introduce RecreationWorld, a five-platform framework built around recreation: given a running reference, an agent must discover its behavior and build a faithful implementation with no prescribed workflow. RecreationWorld provides reproducible environments on Ubuntu, macOS, Windows, Android, and Web, plus a unified harness with native GUI control and coding tools. The running reference serves as an oracle for hidden behavioral tests, providing execution-grounded rewards. We scale trajectory generation with high-quality open-source applications. Models trained on these trajectories improve across five out-of-distribution coding and hybrid computer-use benchmarks and more frequently verify their rendered outputs, providing evidence of transfer beyond recreation. For held-out evaluation, we introduce RecreationBench, comprising 250 diverse tasks across domains and platforms. Reference-grounded programmatic and visual assertions cover action-conditioned outcomes at multiple interaction depths; each is validated on the reference and by human reviewers before the suite is frozen for automatic scoring. GPT-6 Astra leads at 58.1% overall, but passes all programmatic tests on just 2.8% of tasks. Agents reproduce static interface structure more reliably than interactions and computed outputs, while generated applications remain smaller and more monolithic than their references. We release the benchmark, environments, and test suites.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新