让观众改写剧情,AI 终于知道该看哪了
互动视频生成一直有个没说破的毛病:观众说「让主角去拿伞」,AI 只盯着这句话,却不知道伞在之前的画面里、在哪个位置、和谁有关。这篇把「上下文」拆成三层——先抽象出当前画面里到底有哪些可用的状态和关系,再把观众的新指令投影成具体的生成约束,最后靠规划、验证、修改、渲染四步把约束落实。在三个主流生成模型上,综合评分平均比各自最强基线高 10.3 分。它不是你明天能用的产品,但这是互动视频从「听指令」走向「懂语境」的关键一步。
📄 原文摘要(英文)
Video interactive storytelling enables viewers to actively steer how a video unfolds. However, once we allow viewers to intervene during generation, a new challenge arises: The viewer's request can have latent dependencies on both the grounding source and the current rendered video state. These dependencies may not be explicitly stated in any individual input, but emerge only when the source, rendered history, and new viewer intent are considered jointly. Existing interactive video generation systems primarily emphasize following viewer instructions, while source-grounded video generation methods focus on aligning generated content with an external narrative or knowledge source. This leaves a fundamental question underexplored: What context should a generation model ground on during interactive continuation, and how can heterogeneous, unstructured inputs be transformed into such grounding context? In this work, we formulate contextual grounding as the process of transforming heterogeneous input context into an executable constraint model for video generation. To address this challenge, we introduce VIS-Ground, which performs Structured Context Abstraction to recover grounded states and cross-context dependencies, Generation Constraints Induction to project relevant dependencies into candidate-specific constraints, and Constrained Video Generation to enforce these constraints through planning, verification, revision, and rendering. Across three video generation backbones, VIS-Ground consistently achieves the highest overall composite score, reaching an average absolute improvement of 10.3 points over the strongest per-backbone baselines. Detailed analysis further shows gains across both narrative and knowledge grounding, and reveals remaining challenges in dependency extraction, and faithful realization during video rendering.