把AI生成视频变成可编辑的3D世界
现在的AI生成视频是一次性的：你给一句话，它吐一段画面，想改个镜头或换个物体位置，只能重新生成，之前的设定全丢。这篇把思路反过来——先让AI搭出一个持久的3D场景，里面每个物体、每台相机都是独立可调的零件。你改一个地方，其他部分保持原样，就像在游戏编辑器里摆弄场景，而不是每次从头画。它把2D生成的内容提升成连贯的3D世界，用双视角初始化保证一致性，编辑时只更新变化的部分，镜头规划也通过渲染反馈来验证可行性。对做电影预演、游戏原型、建筑可视化的人来说，这是把AI从“一次性生成器”变成“可迭代工作台”的关键一步。
📄 原文摘要(英文)
Previsualization is an intermediate layer between ideas and production in film, games, architecture, and urban design. It lets creators iteratively refine scenes, actions, cameras, and spatial-temporal dynamics. Yet existing generative methods rely on simple prompts to jointly control all of these factors through one-shot image or video synthesis, offering weak controllability and limited support for iterative editing. Fundamentally, a world comprises multiple elements with geometry, appearance, and other attributes, together with cameras. Different frames are produced through local modifications or recombinations of this shared state, which is otherwise largely reused. Therefore, we argue that the missing component is an explicit and persistent working state. To address this, we present StateFlow, a state-centric framework for generative previsualization. Rather than generating videos in one shot, StateFlow uses an editable 3D world to organize scene structure, evolution, and cameras, while off-the-shelf video models enhance visual quality when higher fidelity is desired. This world is maintained as a persistent structured 3D state of scene elements and camera configurations, serving as the core working representation for previsualization. Built on this insight, StateFlow has three stages to construct, evolve, and access the world state. State construction lifts generated 2D content into a coherent 3D world through prior-guided, conflict-aware dual-view initialization, while State evolution translates user intent into structured state transitions while preserving world memory, avoiding full-scene regeneration for each edit. State access uses render-feedback reflection to refine camera plans into visually feasible trajectories, avoiding reliance on VLM semantics alone. Experiments show that StateFlow produces high-quality 3D worlds for video creation and game-like prototyping.