AI视频生成终于能让人和物体“互动”了
现在的AI视频生成，要么只能生成一个人，要么只能生成一个物体，人和物体之间的互动（比如人拿着杯子、人穿衣服）总是穿帮——物体像贴上去的，动作不自然。这篇论文让AI能同时记住“这个人”和“这个物体”，并且让它们按照真实物理规律互动。核心做法是：先用多模态大模型理解参考图片中人和物体的关系（比如“手握着杯子”），再把这些语义信息注入到视频生成的每一帧中，同时用特殊编码区分“人的特征”和“物体的特征”，避免混淆。结果：生成的视频中，人拿杯子、穿衣服、甚至和抽象物体（如logo）互动，都更真实、更一致。它不是你明天就能用的工具，但如果你做电商视频、虚拟试穿、或者需要生成“人使用产品”的广告素材，这是未来最可能落地的方向。
📄 原文摘要(英文)
Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter-subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra-subject references (e.g., OCR maps, multi-view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference-level relationships without compromising the controllability of text encoders or incurring costly re-alignment. Specifically, we introduce global multimodal guidance within self-attention to better align MLLM-derived semantic features with VAE tokens. Furthermore, we propose modality-reference embedding to differentiate tokens from MLLM features and VAE tokens and associate intra-subject reference image tokens. Extensive experiments validate that our method achieves state-of-the-art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie-page.github.io/