AI 自学刷题，越刷越像死记硬背
让 AI 自己读教材、自己刷题，它到底是在「学会」还是在「背题」?这篇论文造了一个物理题库来直接测这件事：同一批训练材料，一部分题和教材高度相似(应用集)，另一部分直接上奥赛难度(迁移集)。结果很扎心：几乎所有方法在应用集上进步明显，一到迁移集就几乎原地踏步；更狠的是，把同样的材料直接塞给 AI 当提示词，它立刻就能解出难题——说明材料里信息是够的，是 AI 自己的「消化」过程出了问题。所有方法还都撞上算力天花板，再喂更多算力也不涨。结论是：瓶颈不在数据和算力，而在方法本身。这不是你明天能用上的东西，但它给「AI 到底会不会举一反三」这个老问题，第一次画出了一条清晰的测量线。
📄 原文摘要(英文)
Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolution method should share the same property, that is autonomously learning from raw training material for transferable problem-solving capability. However, we still lack a direct measurement for it. We introduce StudyBench, a controlled physics benchmark that directly measures how efficiently a self-evolution method converts training material into capability. We organise the test set into an Application Set, consisting of difficult textbook problems and evaluating absorption ability, and a Transfer Set, consisting of olympiad-level problems and evaluating transfer ability. Benchmarking representative self-evolution methods across three base models, we find that improvements on the Application Set rarely translate to the harder Transfer Set. A guidance ablation exposes a Guidance Gap: even the strongest method closes only a small fraction of what the same material unlocks when supplied as in-context guidance. Besides, every method hits a Compute Plateau, saturating well before exhausting its compute budget. The remaining gap is therefore a method problem rather than a data or compute problem. By offering a clean and controlled benchmark, StudyBench turns self-evolution progress from an open-ended pursuit into a measurable target for future research. Our code is released at https://github.com/thunlp/StudyBench.