4比特量化连循环层都敢压，27B模型反而更小更快
大家一直以为，大模型里那种「用固定大小的记忆把整个上下文浓缩起来」的循环层，是最怕压缩精度的——误差会在一次次递归里滚雪球。所以社区做 4-bit 量化时，总是偷偷给这半边留 8-bit 甚至 16-bit。这篇直接反着来：把 27B 混合模型里全部 496 个线性层，包括 48 个 Gated DeltaNet 循环层，统统压到 NVFP4 4-bit。结果在长上下文困惑度、数学、代码、检索等一堆基准上，和原始 BF16 精度几乎没差(5 项平均只掉 0.52，在随机噪声范围内)，模型体积缩到 17.5 GiB,prefill 还快了 14-19%。更反直觉的是，他们拆开看误差来源：最该脆弱的「门控」投影其实最不敏感——因为 softplus/exp 和 sigmoid 这类激活函数把约 11% 的矩阵乘法误差压成了约 2% 的输出误差；而 delta-rule 的递归本身会把注入的噪声压在一个平坦平台上，几百步内就忘掉一次状态冲击，因为每次写入都沿着当前 key 方向覆盖旧状态。误差不累积，反而被上下文冲淡。这不是你明天能直接用的技巧，但它改写了「循环层必须高精度」这条默认规则——以后做端侧小模型，整机 4-bit 不再是禁区。
📄 原文摘要(英文)
Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes the context in fixed size. Early community 4-bit quantizations of Qwen3.8-27B (48 GDN layers, 16 attention layers) left the GDN block in 8- or 16-bit precision -- especially its decay and write-strength gates -- on the intuition that errors in a recurrence accumulate over long contexts. We test that intuition by building Minima: NVFP4 W4A4 on all 496 linear layers, GDN included. Across perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K, Minima matches BF16 within seed noise (5-task average -0.52) while being the smallest (17.5 GiB) and fastest-prefill (+14-19%) recipe we compare, and its 32K perplexity gap shrinks with position. A four-part mechanism study explains why: (i) NVFP4's 16-element block scaling localizes the residual stream's extreme outliers, equalizing activation error across layer roles; (ii) the supposedly fragile gate projections are the least sensitive -- softplus/exponential and sigmoid parameterizations compress ~11% GEMM error to ~2% output error; (iii) the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens and forgets a state impulse within hundreds of steps, because each write overwrites the state along the current key direction; (iv) the per-token quantization cost washes out with context instead of compounding. We also repair a global-scale mismatch that arises when per-module-calibrated NVFP4 checkpoints are served by kernels that fuse those modules into one GEMM, and show calibrated FP8 KV-cache scales are performance-free. The result: a practical recipe -- quantize everything, ship KV scales -- and a mechanistic account of why the recurrent half of a hybrid LLM is the easy half to quantize. Checkpoint: https://huggingface.co/minima-ai/mnma_qwen3.8_27b_nvfp4