AI自己写GPU内核,登顶三榜
一个AI系统从零开始,自己搜出了比官方参考快4.4倍的GPU内核,还霸榜了。它拿到的只有每个问题的PyTorch定义,没有参考内核、没有官方分数,纯靠试错。做法是:模型外挂一个档案库,存下每个正确内核和它的「家谱」;一个LLM控制器读档案,挑一个父本或开新设计,再给每个并行子代理指一个方向;对错由模型外的检查判定,不由模型自己说了算。结果在SOL-ExecBench的L1、L2、量化三个公开榜上排第一,235个问题全部有验证通过的内核,209个公开题上平均快4.4倍。同样时间里,它比手写的树搜索在38题中赢了35题,靠的是多做出2.4倍的正确尝试;上千个代理同时跑,一个热重启修复把评估速率提了3.6倍。它真正的发现是:卡住这类搜索规模的不是代理数量,而是可靠测量的速率。这离你明天用上还远,但它指的方向很实在:让AI改进AI,瓶颈在「怎么可信地打分」,不在「堆多少算力」。
📄 原文摘要(英文)
We present CompoundingKernel, a self-improving agent system that writes GPU kernels by search. As of 2026-11-11, its generated kernels rank #1 on the public L1, L2 and Quantization boards of SOL-ExecBench (NVIDIA B200). They have held #1 on L1 since 2026-10-27, albeit a short loss on 2026-11-07. All 235 problems have a site-verified correct kernel; on the 209 public-board problems they are 4.40x faster than the benchmark's reference baseline. The search starts from zero: agents get each problem's PyTorch definition but no reference kernels or official scores. A per-problem archive outside the model holds every correct kernel and its lineage. An LLM controller reads it, picks a parent or a new design, and gives each parallel child agent one direction. Checks outside the model decide correctness. At equal wall time, it beat a coded UCB style tree on 35 of 38 problems by making 2.4x as many correct attempts. Over a thousand agents ran at once, and one warm restart fix raised their evaluation rate 3.6x. We find that the rate of reliable measurements, not the number of agents, limits how far such a search scales. Our code and kernels are available on URL.