模型给自己动手术,越改越强
我们习惯把训练和推理当成两件事:训练一次,冻住,上线,想变强就再训一轮。这篇说,一个 27B 的模型可以自己给自己改权重——没有梯度,没有训练,只靠它自己的推理。给它自己的权重、带评分的答案、以及运行自己的能力,它先诊断自己错在哪,再动手改权重。改完之后,在事先定好的 490 道没见过的题上,准确率涨了 6.6 个百分点;换一次会话再改,也显著变好。它只在自家测试过关时才提交修改,每次覆盖上一次,头八轮下来准确率只升不降。和「推理时多算几遍」不同,这个提升写进了权重里,每答一题不多花成本;和训练不同,它不需要预训练或后训练那套基础设施,一个开放模型加一张 GPU 跑几小时就够。它还能修自己:把一份被弄坏的副本交给它,它定位到损坏的矩阵,用自己的数值重写,全程不读未损坏的权重;只靠文字数学题找到的修法,让这份副本在六个没见过的图像基准上从 7% 升到 70%,达到完好模型的 83%。这个能力随规模出现:同一家族四个尺寸里,2B 不动手,4B 和 9B 把自己改坏,只有 27B 像人一样工作——对比「答完的」和「token 用尽的」两种答案的隐藏状态,削弱区分它们的方向。它不是你明天能用上的东西,但它指向一件事:改进一个语言模型,未来可能是一次对话,而不是一轮训练。
📄 原文摘要(英文)
Language models are trained once, frozen and deployed: improving one again takes another training run, and a prompt, a scaffold or extra computation at inference is gone once the task ends. A frontier model has been shown to edit the weights of a much smaller one. We ask whether a model can rewrite its own weights to make itself better, and find that it can, by its own reasoning, with no gradients and no training; we call this self-rewriting. Given its weights, graded answers and the ability to run itself, a 27B model diagnoses its failure and edits the weights behind it. It surpasses itself: its own edit makes it 6.6 points more accurate on 490 problems it never saw, in a test fixed in advance, and another session's edit also makes it significantly better. It keeps getting better: it submits an edit only when its own tests support it, each edit replacing the last, and over its first eight such sessions its accuracy only rises. Unlike test-time scaling, the gain persists in its weights and adds no cost per answer; unlike training, it needs no pre- or post-training infrastructure, only an open model and hours on one GPU. It repairs a damaged copy of itself: it locates the corrupted matrix and rewrites it with values of its own, never reading the undamaged weights; found on text math problems alone, the repair lifts the copy on six image benchmarks it never saw from 7% to 70% accuracy, 83% of the undamaged model's. And the ability emerges with scale: across four sizes of one family, the 2B model makes no edit and the 4B and 9B models break themselves; only the 27B model works on itself as a human would, comparing its hidden states on answers that finish and on answers that run out of tokens, weakening the direction that separates them. Like a human, a model need not separate training from inference; improving a language model may come to take a conversation rather than a training run.