改图里的文字,终于不用再靠假数据练了
现在的 AI 改照片里的文字,训练时看的全是电脑合成的假图,一到真实世界就露馅——要么糊、要么凭空多出笔画。这篇反着来:不造假的,直接把一张真图拆了当训练素材。做法很朴素:把原图里的文字列打乱、再裁掉一块,文字语义没了,但字体、颜色、背景这些风格还在,模型就从这些真图里学「怎么改字但保留风格」。结果在真实场景上,准确率和真实感都明显超过依赖合成数据的旧方法。它不是你明天就能拿来 P 图的东西,但它指了个方向:与其费劲造更真的假数据,不如让模型直接在真数据上自己找活干。
📄 原文摘要(英文)
Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications. STE typically requires paired images before and after editing for training. However, recent approaches rely heavily on synthetic paired data and complex disentanglement strategies. They struggle to faithfully capture the diversity and complexity of natural scenes, often producing artifacts or hallucinations. In this work, we challenge the common practice and investigate whether STE can be driven by turning each real sample into its own paired supervision. Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self-paired training data, and build a strong baseline (TextRIRO}) upon a conditional diffusion architecture. Our experiments show that combining character-column shuffling with image cropping (Shuffle&Crop) effectively destroys text semantics while preserving key style cues, enabling the model to learn editing behaviors directly from the real text image. With this concise yet real-oriented design, TextRIRO achieves promising performance in both accuracy and visual realism across real-world benchmarks, substantially mitigating the synthetic-to-real domain gap. Furthermore, we leverage TextRIRO to build TextRIRO-3M, a large-scale realistic dataset for long-text recognition by concatenating edited and original images. Experimental results show that training recognition models with TextRIRO-3M significantly improves recognition accuracy, demonstrating that TextRIRO provides highly useful training resources for downstream tasks. Overall, TextRIRO lifts STE to realistic, reliable, and widely applicable new levels. Code is available at https://github.com/YesianRohn/TextRIRO.