给AI一张骨架图,它就能补出整段视频
现在的AI生成视频,你只能给文字提示,它自己决定画面里的人和物怎么动。这篇反着来:你先把场景的布局、动作、镜头位置定好——哪怕只是一张骨架图、一段线条画、一堆点云——AI负责把剩下的外观细节补成真实视频。做法是把这些粗糙结构当成普通文字描述喂给一个冻结的大模型,只加两个轻量适配器,就能在4步内出片,比原来快12倍,还能实时渲染768p视频。它不是你明天就能用的工具,但方向很明确:以后做游戏、做动画,可能不再需要手动调材质和光影,给个草稿,AI帮你填完。
📄 原文摘要(英文)
Game engines and coding agents can specify the layout, motion, and camera of a scene, yet turning such coarse structure into realistic video requires a renderer that follows the structure and completes the appearance it leaves open. Existing renderers are tied to the representation of one engine, need tens of denoising steps or a costly distillation of their own, and learn from conditions that are not aligned with their target videos. We present SolarRenderer, a structure-conditioned video renderer that adapts a frozen MiniMax-H3 33B backbone with two lightweight LoRAs. Structural videos enter the native reference pathway of the backbone at the positions of the frames they describe, without a type identifier and described only in plain text, so that a single control LoRA follows arbitrary structural representations. Starting from a pretrained four-step LoRA, structure-guided distribution matching renders in four steps, 12.3 times faster than 50-step sampling, while preserving the learned control. A block-causal variant extends the same design to streaming and renders 768p video in real time. To train the model, we build a frame-aligned corpus of 136K clips from game engines, simulators, and YouTube videos. Experiments on an in-distribution test set and a generalization benchmark of 16 structural representations show that SolarRenderer achieves state-of-the-art results across both, and generalizes to 11 unseen representations, including pose skeletons, drawings, and point clouds.