全景图生成:AI终于能理解球面几何了
现在的AI生成全景图,经常把地平线搞弯、柱子变歪,因为模型不懂球面几何。Canvas360在训练时加了一个“平行深度生成”和“速度循环填充”的步骤,让AI先学会全景图的扭曲规律,再去做风格迁移、修补、扩图等任务。它用100万对高质量全景数据训练,在专业评测指标FAED上表现突出。虽然你明天用不上,但它解决了全景生成“看着像但一细看就穿帮”的核心问题。
📄 原文摘要(英文)
In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/