AI解题像人一样：少换思路，深挖一条
大模型做数学题时，我们一直盯着它“想了几步”，但真正决定对错的不是步数，而是它“怎么想”。研究者用数学教育里的两个概念拆解AI的推理过程：一是它把题看成代数还是几何（语义空间），二是它具体用了什么招（比如化简、倒推）。结果发现，AI和人类一样，集中精力在少数几个思路上深挖，比到处乱试更容易做对；而强化学习训练会让模型变得更“一根筋”，只爱用少数几种招数。研究者反过来用这个发现去训练模型，鼓励它多用不同解题招数，准确率确实提升了。这不是你明天能拿来用的工具，但它给了我们一个更靠谱的视角：判断AI会不会做题，别只看它写了多少步，要看它有没有像人一样“想得深”。
📄 原文摘要(英文)
Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their reasoning remain underexplored. We introduce SHAPE, a framework that analyzes Chain-of-Thought (CoT) trajectories through two lenses developed in mathematics education: (1) semantic spaces: the model's evolving mathematical interpretations of a problem (e.g., algebraic, geometric), and (2) heuristics: the specific mathematical actions taken within those spaces (e.g., simplifying the problem, working backward). We first use SHAPE to analyze the reasoning patterns of various models. Our findings reveal that the mathematical heuristics employed by a model better explain final answer correctness than traditional CoT features. Furthermore, models are likely to reach correct solutions by concentrating their reasoning effort within a few semantic spaces rather than exploring many disparate ones -- a pattern consistent with human behavior. Next, we utilize the SHAPE lens to evaluate whether post-training truly enhances mathematical proficiency. We find that reinforcement learning induces mode-seeking in heuristic usage. Lastly, we post-train LLMs by promoting diverse heuristics and demonstrate its effectiveness in improving accuracy. Overall, SHAPE provides a theoretically-grounded diagnostic framework for decoding LLM reasoning and offers a new path toward post-training LLMs for math reasoning. The code for our model is available at https://github.com/holi-lab/SHAPE-of-CoT