AI画图时，那些“废话”token竟在偷偷保命
你给AI一句提示词，它生成一张图。但你可能不知道，提示词里那些“a photo of”、“in the style of”之类的结构模板词，在AI内部其实成了“隐形语义寄存器”——它们自己几乎不携带提示信息，却像吸铁石一样吸引图像token的注意力，暗中维护着物体的身份。研究者用因果分析拆解了DiT模型（当前主流文生图架构），发现提示语义先注入图像latent，再被模板token读回去，而不是直接从提示词传递。基于此，他们发现那些最关注提示词的注意力头反而是冗余的，剪掉20%的注意力计算量，生成质量几乎不掉。这不是你明天能用的技巧，但它揭示了AI画图时一个反直觉的内部机制：输入时承载语义的token，未必是生成时维持语义的那一个。
📄 原文摘要(英文)
Text-to-image diffusion transformers (DiTs) jointly process text and image tokens, yet their internal computation during denoising remains poorly understood. We introduce a causal interpretability framework for modern large-scale DiTs that combines attention decomposition with targeted interventions across token spans, heads, and layers. Using it to separate prompt-content tokens from structural template tokens, we find that the structural tokens carry little prompt-specific information at the encoder output. Yet surprisingly, they emerge as dominant image-to-text attention sinks and causally maintain object identity inside the DiT, acting as implicit semantic registers. We show that they acquire this identity indirectly, with prompt semantics first injected into the image latents and then read back into the template tokens rather than transferred directly from the prompt tokens. Inspired by the above findings, we design a training-free pruning rule for DiTs. Heads that attend most strongly to prompt tokens are dispensable, and pruning them removes 20% of attention FLOPs with only a 1.4-point drop on GenEval. We further reveal how generative computation in DiTs is organized across heads and depth, separating semantic routing from visual synthesis and progressing from identity formation to propagation and refinement. Our work not only reveals that the tokens encoding semantics at input need not be those that maintain it during generation, but also provides a causal view of internal mechanisms in DiTs.