AI Pulse
📄 论文解读

视频生成提速2倍:只算关键注意力

视频生成慢,一大原因是模型要处理海量像素块,每个块都要跟所有其他块算“注意力”,计算量爆炸。这篇论文发现,其实大部分注意力是浪费的,只有少数关键块真正重要。他们搞了个新方法,在计算过程中动态判断哪些块重要,只算这些,不重要的直接跳过,而且跳过的块还能用近似值补上,不影响画质。结果:视频生成和编辑速度分别提升2.1倍和2.3倍,画质几乎不变。这不是你明天能用的工具,但它是视频生成走向实时的重要一步——未来你刷短视频、做AI视频时,背后可能就靠这个省时间。

📄 原文摘要(英文)

Diffusion transformers are essential for high-fidelity video generation, but long token sequences make attention a dominant inference bottleneck. Training-free dynamic sparse attention alleviates this bottleneck by computing only selected key-value blocks, yet existing methods struggle to sparsify attention both efficiently and accurately for two reasons: (1) Rigid, unpredictable, and costly routing: selecting a fixed fraction of top-ranked blocks by proxy score imposes fixed budgets, whereas retaining blocks to reach a target cumulative proxy probability mass yields dynamic but potentially imbalanced budgets; both incur non-negligible overhead from computing and materializing proxy scores. (2) Lossy keep-or-drop sparsification: unselected blocks are discarded entirely, degrading accuracy under aggressive sparsity. These limitations motivate cheaper dynamic-budget routing while limiting accuracy degradation. In this paper, we introduce training-free Sol-Attn (Sparsifying online attention), which unifies dynamic routing, sparse computation, and approximation correction in a single online-softmax pass, achieving a better accuracy-efficiency trade-off in sparse attention. The core of Sol-Attn is on-the-fly block thresholding with proxy-score reuse, which selects critical blocks by comparing block proxy scores against a threshold during online softmax. This design enables dynamic yet controllable block budgets without materializing the proxy map, while directly reusing the proxy scores of unselected blocks to approximate their contribution. Experiments across image and video generation tasks show that Sol-Attn advances the quality-efficiency frontier of training-free sparse attention, delivering 2.1 times and 2.3 times end-to-end speedups for video generation and editing, respectively, while preserving visual quality.

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部