一个模型搞定文生图、局部改、全局改,还不打架
现在的图像生成模型,文生图、局部编辑、全局编辑这些能力往往互相冲突——你让模型既能画图又能改图,它反而哪样都做不好。这篇论文提出 DanceOPD,核心思路是:给每个能力(比如“画一只猫”“把猫换成狗”“把背景变暗”)单独定义一个“速度场”,然后让模型在生成过程中动态选择走哪个场,并用自己当前的状态去学习。这样不同能力各走各的路,互不干扰,还能吸收像无分类器引导这样的操作。实验证明,它能在不牺牲基础文生图质量的前提下,显著提升编辑能力。这不是你明天就能用的工具,但它为“一个模型什么都会”提供了一条靠谱的技术路径。
📄 原文摘要(英文)
Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing. However, these capabilities are rarely naturally aligned and often conflict. For instance, editing tends to degrade T2I performance, while global and local editing interfere with each other. Consequently, effectively composing these capabilities has become a central challenge for image generation model training. To tackle this, we introduce DanceOPD, an on-policy generative field distillation framework for flow-matching models that routes each sample to one capability field, queries one low-noise student-induced state, and trains with a simple velocity MSE objective. With each capability source defined as a velocity field over the shared flow state space, the student learns from fields queried on its own rollout states to compose expert capabilities. This formulation also absorbs operator-defined fields such as classifier-free guidance. Comprehensive experiments on T2I, editing, realism-field absorption, and CFG absorption show that our approach improves multi-capability composition, strengthening target capabilities while preserving anchor generation quality. We believe this work establishes a practical route for generative field distillation in flow-matching models.