长文本AI推理提速47倍，但别急着用
大模型处理长文本时，最耗时的不是生成回答，而是先读一遍所有内容——这个叫prefill的阶段，计算量随文本长度平方增长。FlashPrefill V2把注意力机制改成稀疏的：只让模型关注真正重要的部分，忽略无关内容。它用了三项改进：一个修正项让稀疏化后精度不掉太多；一套GPU底层优化让计算跑得更快；还直接兼容了主流推理框架。在128K上下文长度下，比当前最常用的FlashAttention-2快最多47倍。但诚实说：这是前沿研究，不是明天就能装进你用的AI产品里的东西。它证明了稀疏化这条路能走通，但离稳定部署还有距离。
📄 原文摘要(英文)
Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase. Our previous work, FlashPrefill, mitigates this cost through instantaneous pattern discovery and max-based dynamic thresholding; however, it remains an algorithmic prototype that is still distant from production deployment. In this paper, we present FlashPrefill V2, which evolves FlashPrefill from a prototype toward practical long-context serving along three dimensions. First, we introduce a mean correction term that effectively suppresses the approximation error, keeping performance degradation manageable even at extreme sparsity levels. Second, we redesign the sparse attention operator with PackGQA memory access, warp specialization, and pingpong pipelining, fully aligning with the latest FlashAttention-3/4 implementations and supporting FP8 inference to meet practical quantization requirements. Third, FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs---among the most widely deployed inference accelerators---demonstrate that FlashPrefill V2 delivers up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context length under FP8 and BF16 precision, respectively, and, in FP8, still achieves a 30.49x speedup against an FA3/4-aligned dense baseline.