把图像生成器改造成全能视觉识别器
图像生成模型不只是会画画，它内部已经存着对真实世界的理解。这篇把这件事做成了人人可复现的配方：不训练新模型，而是把开源的 4B/9B 图像生成器改造成能分割、能测深度、能按描述找物体的多任务视觉系统，训练数据全部来自公开数据集。他们发现几个关键设计会互相打架——单独调好一个，合起来反而变差，所以配方比零件更重要。结果在城市场景的零样本分割上超过了原版未公开的 Vision Banana(73.9 对 69.9)，深度估计也追平当前最强。还顺手加了个几乎零成本的技巧：让模型在几种不同的「颜色对应类别」图例下各分割一遍，再投票合并，不用重训就把分割精度最多提 8.7 和 15 个点。做视觉模型、想用生成器替代专用识别网络的研究者和工程团队，这是可以直接拿去跑的完整方案；对普通人，它意味着这类「一个模型看懂多种视觉任务」的能力正在从闭源走向公开可复现。
📄 原文摘要(英文)
Recent work suggests that pretrained image generators encode real-world understanding beyond image synthesis. Vision Banana exploits this by framing vision tasks as conditional image editing. However, its data, weights, code, and implementation details are not public. We present Open Vision Banana (OVB), a fully open and reproducible recipe for converting pretrained generators into unified multi-task perception systems. Our analyses identify the design choices that matter most for generalization, including color encoding, prompt design, parameter adaptation, and task balancing. These choices interact, so individually helpful choices can hurt when combined with others. Starting from 4B and 9B parameter open-weight base models and using 2.1M-3.8M training pairs from public datasets, OVB surpasses Vision Banana's reported Cityscapes zero-shot performance (73.9 vs. 69.9 mIoU) and rivals state-of-the-art affine-invariant depth estimators. Its non-zero-shot variants come within five points of Vision Banana on referring and reasoning segmentation but remain further behind on instance segmentation. All models largely retain the base model's image-editing abilities. With legend ensembling, we introduce a test-time scaling method for generator-based perception. It segments an image under several class-color legends and fuses the results by majority vote, correcting unstable color-class bindings and improving mIoU by up to 8.7 points on Cityscapes and 15.0 points on COCO-Object without retraining. We release the dataset construction and processing pipeline, the training and evaluation code, as well as the final, intermediate, and ablation checkpoints. Project website: https://openvisionbanana.github.io