不训练不提示，直接改权重修好AI
修好一个大模型的推理毛病，以前要么重训、要么改提示词；这篇直接动它权重里的几个比特。MIT 那套「权重空间编程」让一个前沿 AI 智能体读权重、写修复，代价是 158 轮对话、8970 万 token。Tree-Ising 把这件事变成模型自己 BF16 权重上的稀疏离散优化：用前向统计定一个目标，削弱「绕圈」的方向、加强「收尾」的方向，同时保住正常 token 的输出；允许的改动只有符号翻转和指数移位，且不破坏共享的指数与尾数。目标被精确拆成每行一个独立的 QUBO(Ising)问题，在 GPU 上用模拟退火或量子启发的模拟分叉求解。效果：同样两层上 MATH-500 拿到 54.6%，超过智能体改法的 51.7%，基座只有 7.9%;改动权重少 13%–17%，比特翻转少 2.2–2.3 倍，只要 4 次前向、0 个 LLM token，退火方案约 0.9 美元 GPU 时间。换到 Qwen3.5-2B，同一套配方把 GSM8K 贪心准确率从 21.6% 拉到 69.4%。它不是你明天能上手的东西——目前是研究级的概念验证，连八变量子问题都只是拿到 IBM 量子处理器上跑通。但它指出的方向值得记住：修模型可能不再需要海量算力和数据，而是像修电路一样，找到那几个错的比特。
📄 原文摘要(英文)
Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.