AI看图时，偷偷在用相机指纹
AI 看图时，你以为它在认物体，其实它在偷看照片的「相机指纹」——处理算法、拍摄设备留下的像素级痕迹。研究者发现，大规模训练让模型把这种看不见的元数据当成了预测线索，而且训练数据里这种关联越强，模型越依赖它；一旦照片来源变了，性能就明显掉。好消息是，他们找到了在训练前和训练后都能削弱这种依赖的办法，还不牺牲正常任务的表现。这甚至解释了为什么有些模型特别擅长识别 AI 生成的图——它们靠的正是这种痕迹。
📄 原文摘要(英文)
Deep vision models exploit shortcuts, relying on cues that correlate with supervision signals. Prior work has focused on visible biases, such as object-background or texture correlations. We identify a different source of shortcut learning: invisible metadata traces embedded at the pixel level, for metadata such as image processing and photo acquisition. We hypothesize that large-scale semantic supervision, whether through categorical labels (ImageNet) or billion-scale captions (LAION), naturally induces metadata-semantics correlations during pretraining, leading models to convert low-level signals into predictive features. By introducing controlled metadata-semantics correlations, we show that stronger ones produce systematically higher sensitivity to metadata traces and larger performance degradation under metadata distribution shifts. We further explore mitigation strategies applied during and after pretraining that reduce sensitivity not only to targeted metadata but also to unseen ones, without sacrificing performance on downstream tasks. Metadata sensitivity also has a positive side: it partly explains the strong generated-image detection ability of some encoders, while its mitigation can improve out-of-distribution generalization. Code: https://github.com/ryan-caesar-ramos/visual-encoder-traces