AI没有标准答案也能自我纠错
现在的大模型数学推理在考试前还能临时抱佛脚：不用标准答案，自己生成多个解法，然后把一致的对、不一致的错，分开处理——对的多学，错的挨罚。这套方法在五个竞赛级题库上追平了有标准答案监督的训练效果，一个中等模型做题正确率从38%提到45%。它说的是：AI 有能力在没有外部正确答案的前提下，靠内部共识持续变强。这不是你明天能装进 app 里的功能，但它是通往「AI 自己越用越聪明」的一种可能。
📄 原文摘要(英文)
Recent prominent post-training methods, such as Reinforcement Learning (RL) and On-Policy Self-Distillation (OPSD), have driven rapid progress in mathematical reasoning for large language models, yet their reliance on ground-truth labels precludes test-time training (TTT). Replacing ground truth with majority-vote pseudo-labels is a natural alternative, yet it is fragile: an incorrect vote corrupts the teacher and misleads every token. We observe that this failure mode is asymmetric: rollouts that disagree with the pseudo-label are typically wrong regardless of whether the vote itself is correct. Building on this observation, we propose Test-Time Policy Optimization (TTPO), an asymmetric objective that distills agreeing rollouts via OPSD and penalizes disagreeing rollouts with Grouped RL. Token-level selection further refines both branches: distillation down-weights already-converged positions, while RL penalizes only confident errors. Both updates remain well-grounded even under frequent pseudo-label errors, and majority-vote routing yields tighter self-supervision as the model improves. Without any labels, TTPO matches label-supervised OPSD on five competition-level benchmarks, raises Qwen3-1.7B from 38.0% to 45.2% in TTT, yields +25.2% to +36.4% without thinking, and shows strong cross-task generalization.