AI Pulse
📄 论文解读

给扩散模型戴一副降噪眼镜

扩散模型在像素空间里训练时,高频区域其实全是噪声,但模型却要自己摸索哪里是信号、哪里是噪声,白白浪费算力。这篇论文发现了一个规律:在自然图像里,信号和噪声的边界会随时间变化,于是他们设计了一个叫 Spectral Forcing 的滤波器——在模型看图像之前,先根据当前时间把高频噪声切掉,只让低频信号通过。这个操作不增加参数,却让模型在 ImageNet 和文本生成图像任务上 consistently 提升 FID 和 Inception Score。它不是你明天能用上的技巧,但它揭示了一个趋势:与其让模型自己猜,不如把物理规律(比如噪声分布)直接写进架构里。

📄 原文摘要(英文)

Pixel-space diffusion models are trained on full-bandwidth noisy images, yet the useful signal available to the denoiser is strongly frequency dependent. Under rectified-flow diffusion and natural-image power-law spectra, the per-band data-to-noise contour k^{*}(t) = (1-t)^{-2/α} separates a signal-bearing low-frequency region from a noise-dominated high-frequency region at each time t. We show that this implicit coarse-to-fine structure is not merely descriptive: it induces a capacity-allocation problem. A standard pixel-space denoiser must discover the moving bandwidth boundary internally and can spend computation on frequency-time regions where the optimal prediction collapses to deterministic baselines rather than data-distribution modeling. To make this boundary explicit, we introduce Spectral Forcing, a parameter-free, time-conditional 2D-DCT low-pass operator applied to the noisy input before the patch embedder. Its cutoff expands monotonically with the diffusion time and becomes the identity at the data endpoint. Through controlled synthetic experiments, we identify the regime in which the operator is beneficial: coarse patch tokenization and data whose high-frequency content is predominantly noise rather than essential signal. On ImageNet-256 with JiT-700M/32, Spectral Forcing consistently improves both FID and Inception Score across different training epochs, demonstrating robust gains throughout training; at finer tokenization, the spectral forcing is still competitive. We further insert the unchanged operator into SenseNova-U1, a unified text-to-image model, where it improves DPG-Bench and GenEval, showing that the input-side spectral prior transfers beyond class-conditional generation. These results suggest a route to capacity-efficient pixel-space diffusion by showing the signal and hiding the noise.

arXiv 原文

订阅 AI Pulse

每天 09:00 · 14:30 · 19:30 更新