AI画图可以不要解码器了
生成图像和视频的 AI 一直有个隐藏的中间商:模型先在压缩的「潜空间」里画,再靠一个解码器把结果翻译回像素。这篇把这个中间商砍了——让模型在最后一步直接吐像素,而且不牺牲画质。做法很巧:不推翻原来的去噪流程,只微调最后几层,另加一个专门负责「最后一步出像素」的头,原头继续管采样轨迹,两头分工。结果在图像和视频生成上都成立,视频里细节扭曲和闪烁明显减少,形变误差降了 23%;连本来就没有解码器的像素模型也被这套分工救了一把,纹理更细,FID 从 1.55 降到 1.43。对普通读者,这不是明天能上手的功能,但它指向一个趋势:生成模型正在把「翻译」这一步内化掉,同样的算力换更少的失真。
📄 原文摘要(英文)
Latent diffusion models rely on a decoder to map denoised latents to pixels, adding complexity and inference cost. We introduce Decoding Without a Decoder (DWD), which lets a finetuned diffusion transformer produce final pixels directly through a linear head. Unlike previous methods, where the denoising output leads to final pixels, DWD separates the two roles: the original denoising head continues to drive sampling, then the new DWD head predicts the pixels at the final step. We extend pretrained latent models with DWD efficiently by finetuning only the last few transformer blocks, training the DWD head with pixel quality losses while anchoring the denoising head to the pretrained model to preserve the sample trajectory. Across latent image and video generators, DWD removes the decoder and reduces inference cost while maintaining or even improving pixel quality. On Wan video generation, it visibly reduces detail distortion and flickering, lowering warping error by 23%. Remarkably, the same separation also improves pixel diffusion models that do not even use a decoder: a final-step DWD head produces substantially better texture detail in text-to-image generation, and improves ImageNet FID from 1.55 to 1.43, a new state of the art among pixel flow models.