游戏AI当老师：教大模型学会“走一步看三步”
大模型在复杂游戏里常“顾头不顾尾”——赢了不知道哪步关键，输了不知道哪步致命。研究者让游戏求解器（能算出最优解的AI）当老师：它每走一步，就根据局面价值变化给一个“这一步值多少分”的即时信号，而不是等到终局才给个输赢。这相当于把求解器的“直觉”蒸馏进大模型，让模型学会像高手一样判断每一步的长期影响。在推箱子、扫雷、华容道等游戏中，这种方法全面碾压现有训练方案，甚至在未见过的难度和零样本任务（如ALFWorld、WebShop）上也表现最佳。它不是你明天能用上的，但指向一个方向：让AI从“死记硬背答案”转向“理解每一步的因果”。
📄 原文摘要(英文)
Training large language models (LLMs) to act in long-horizon games is a promising step toward generalist decision-making, yet reinforcement learning with verifiable rewards (RLVR) relies on sparse final rewards that reveal little about which decisions determine success. Denser process signals could supply this missing turn-level credit, but existing sources are hard to keep both cheap and accurate. We observe that changes in a game solver's state value reveal whether an action advances the state toward success. Building on this insight, we propose CAST (Credit Assignment from Solver Teachers), which converts these value changes into solver advantages and injects them into RLVR as turn-level signals. We further show that, under a soft-optimal solver assumption, maximizing the solver advantage is equivalent to on-policy distillation from the solver, requiring only scalar values rather than teacher logits. Across Sokoban, Minesweeper, and Rush Hour, CAST outperforms all trained baselines on every game under both in-domain and unseen-difficulty evaluation and achieves the highest average zero-shot performance on ALFWorld and WebShop. Our code is available at https://github.com/Wloner0809/CAST.