AI Pulse
📄 论文解读

AI 管足球俱乐部 20 年,大模型全活下来了

让 AI 当足球俱乐部经理,连干 20 年、做三四百次决策,15 个前沿模型全都撑到了最后,而只会照脚本走的基线对手大多中途倒闭。但真正拉开差距的不是模型大小、价格或算力,而是管理行为:高分模型会在赛季末减少见效慢的投资、让钱动起来而不是躺着、提前续约;token 花多少反而和成绩无关。没有模型能从几百次被拒的报价里学会市场的隐藏价格,自我管理的记忆也都会失效——要么档案只增不查,要么计划年年推翻。它不是你明天能用上的东西,但这是第一次让 AI 在长期、有对手、后果累积的环境里接受同场考验,而且结果说明:长局里,行为比算力更值钱。

📄 原文摘要(英文)

Language model agents now execute bounded tasks reliably. Whether they can sustain effective decision-making over long horizons, where actions have cumulative consequences and the environment responds to their choices, remains largely unmeasured. FM-Bench (Football Management Benchmark) measures this. An LLM agent runs a football club for 20 in-game years through 26 tools and roughly 340 to 400 decision stops. It drafts a squad on the same budget as every rival, trades players, negotiates contracts, invests in facilities and youth, sets lineups, and answers to a board that can fire it, while a deterministic engine accumulates every year into one final score with no LLM judge or human rater. The solo track plays each of 15 frontier models against a frozen scripted world, and the Arena places the same models plus a scripted anchor in one shared 20-year world; to our knowledge, the first head-to-head evaluation at this scale. We measure six behavioral capabilities behind the score. Across three seeds, all 15 models complete every horizon while the blind scripted baselines die out in most of theirs, and claude-fable-5 tops the solo board on mean score and the Arena, where the title nonetheless rotates among ten models. Neither scale, price, nor vendor predicts the order; the order settles only late in the horizon, and the best first-play human lands only at the bottom of the model board. What separates the models is managerial behavior rather than computation. Higher-scoring models reduce slow-payoff investment near the end, keep cash invested rather than idle, and open renewals well before the deadline, while token spend predicts nothing. No model learns the market's hidden prices from hundreds of rejected bids, and self-managed memory fails in two opposite modes: an archive that only grows or a plan rewritten every season. Code is available at https://github.com/Analogy-AI/fm-bench.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新