让AI解题时思路更野但不跑偏
大模型做数学题时，需要生成多种解题路径来找到正确答案。但现有方法要么生成大量只是换种说法的重复路径，要么加入随机噪声导致语义混乱。这篇论文提出一种新策略：在模型内部表示（embedding）层面，把当前词和它最相似的几个邻居词混合，而不是随机加噪声。这样生成的解题路径既多样又保持语义连贯，在数学推理基准上稳定提升，且能泛化到未见过的任务。它不是你明天能用上的，但指向一个关键方向：让AI探索时更聪明，而不是靠蛮力。
📄 原文摘要(英文)
The success of Large Language Models in mathematical reasoning relies heavily on the generation of diverse and valid solution paths during the rollout phase. However, current rollout techniques face a fundamental trade-off: token-level sampling often yields redundant trajectories that differ only in rephrasing, while embedding-level methods utilizing random noise frequently disrupt semantic consistency. To resolve this, we introduce N-GRPO, a novel exploration strategy integrated into the Group Relative Policy Optimization (GRPO) framework. Rather than relying on token-level sampling or native embedding-level noise, our approach leverages Semantic Neighbor Mixing. This mechanism dynamically constructs input representations by mixing the embeddings of an anchor token and its nearest semantic neighbors, thereby injecting diversity while strictly adhering to the local semantic manifold. Experimental evaluations on the DeepSeek-R1-Distill-Qwen models across different sizes show that N-GRPO not only achieves consistent improvements over strong baselines on math reasoning benchmarks but also exhibits robust generalization capabilities on out-of-distribution tasks.