AI 写游戏:能搭骨架,但不会修 Bug
你让 AI 写个游戏,它可能很快给你一个能玩的壳子,但一旦出 Bug,它自己发现不了,也修不好。这篇论文把 AI 写游戏拆成三个环节:从零生成、修 Bug、多轮优化,并为此建了一套评测集。结果很诚实:当前最强的 AI 在「生成可玩基础」上表现不错,但在「自己发现缺陷」和「改完不搞坏原有功能」上几乎不行。它不是你明天能用来做游戏的工具,但告诉你:AI 写代码的下一步瓶颈,不是写,是测和修。
📄 原文摘要(英文)
Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests. Game development is especially demanding because program logic, visual and audio content, interfaces, interaction and playability must function together in one executable artifact. Measuring this capability therefore requires evaluation of both game product and the development process. Existing benchmarks often assess the game development capabilities of LLMs by evaluating the final artifact or an isolated development stage. Our analysis of complete human-agent development trajectories identifies three stages that together span the lifecycle of game development with a coding agent: initial game generation, bug diagnosis and repair, and optimization over multiple turns. Therefore, we introduce GameXpert-Bench, which operationalizes the three lifecycle stages as three complementary benchmark tracks. GameGen evaluates complete game creation from a single request in an empty workspace. GameFix evaluates diagnosis and repair when defects are reported or left for the agent to discover. GameOpt evaluates cumulative optimization through request chains seeded by real development trajectories between users and agents. We evaluate each track using live game interaction, deterministic behavioral tests, or final product criteria with regression checks. The suite contains 97 generation tasks across 11 genres; 100 repair tasks from 50 game levels verified by humans, each with 19-27 injected bugs; and 17 optimization chains with six turns and 102 requests. Across the three tracks, current agents are more reliable at producing playable foundations and implementing explicit requirements than at discovering defects, verifying runtime behavior, and preserving functionality across changes.