AI Pulse
📄 论文解读

AI解说员看陌生游戏,靠查wiki补脑

现在的AI直播解说,训练和测试用的是同一款游戏,换个没见过的游戏就露馅。这篇让解说员学会一件事:遇到叫不出名字的东西,就发截图给一个只读的“研究助手”,后者两秒内从游戏wiki里翻出答案,直播不中断。训练只在《博德之门3》上做,测试却覆盖十个它从没玩过的游戏,实体识别率从0.082提到0.116,而且把“背错游戏设定”的泄漏率从9.9%压到1.0%。它不是你明天能用上的,但方向有意思:让AI承认自己不知道,而不是硬编。

📄 原文摘要(英文)

Streaming video LLMs can now commentate live gameplay, but they are trained and tested on the same games. We study cross-game streaming commentary, in which a model trained on one game commentates games it has never seen, and find that current commentators transfer their timing but not their knowledge of the game world. We propose StreamDuo, which pairs a trainable commentator with a frozen research agent that reads the wiki of the game being played. When the commentator sees something it cannot name, it sends the current frames, and the research agent returns a few wiki entries two seconds later while the stream continues. We train only the commentator, on a single game, in two stages. Information-gain labels teach it to ask only where an answer would help it say what the human caster said, and on-policy self-distillation from a copy of itself that sees the ideal answer teaches it to use noisy answers. Trained on Baldur's Gate 3 and tested on ten unseen games, StreamDuo raises entity recall from 0.082 to 0.116 over Proact-VL trained on the same data, cuts source-game leakage from 9.9% to 1.0%, and achieves the best timing and the highest judged liveliness and overall quality among streaming models.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新