把神经网络权重拆成共享决策路径，压缩方法第一次统一了
压缩神经网络的方法有十几种：剪枝、量化、低秩分解、码本、权重共享……每个都有自己的格式和求解器，像一堆互相不通的方言。这篇把问题重新表述成一个：每个权重都是一串离散决策的终点——符号、2的幂次移位、零门、查共享表。属于同一组的权重共享路径的上半段，每个决策存一次、用一次。于是所有压缩方法都变成对同一个对象的编辑，一个优化器就能在它们之间选择。更妙的是，当决策限定为2的幂次，权重层几乎只剩导线和加法器，适合权重在制造时就固定的芯片。在大型语言模型的五个权重矩阵上，平树结构比GPTQ少用9%到55%的加法器，误差相同。它不是你明天能用上的，但它是压缩领域第一次有了统一的语法。
📄 原文摘要(英文)
We introduce TWIG (Tree-structured Weights with Inherited Gates), a representation in which every weight of a neural network is the end of a path of small discrete decisions: a sign, a shift by a power of two, a zero gate, an index into a shared table. Weights that belong together share the upper part of their paths, and each decision is inherited by every weight below it, so it is stored once and applied once. This simple change has three consequences. Any trained network can be written as a TWIG, and when the decisions are powers of two its weight layers become circuits made almost entirely of wires and adders, suited to chips whose weights are fixed at fabrication. Pruning, quantization, low-rank factorization, codebooks and weight sharing become edits of one object, so a single optimizer can choose among all of them for a whole network. And choosing the decisions is the ground-state problem of a higher-order Boltzmann machine whose interactions follow the paths of the tree, the problem class that annealers and quantum optimizers are built to solve. Today these questions are handled apart: each compression method brings its own format and solver, a circuit with fixed weights needs a dedicated constant multiplier for every weight, and discrete solvers reach only narrow subproblems such as rounding. Weight formats have long been moving toward shared decisions, through group scales, shared exponents, microexponents and sparse weights; TWIG takes this trend to its end, and every common format is a shallow TWIG at no extra cost. On five weight matrices of a large language model, a flat tree whose signed power-of-two terms are chosen per weight for circuit cost, with no group scales, needs 9--15\% fewer additions than GPTQ at equal layer error at 3- and 4-bit accuracy on typical matrices, and 20--55\% fewer on matrices with strong outliers.