AI Pulse
📄 论文解读

一个模型搞定所有视觉任务,不用再换头了

以前做计算机视觉,检测、分割、深度估计各用各的模型,像换镜头一样麻烦。这篇把视觉任务全变成「生成」问题:你给一张图加一句指令(比如「找出所有红色汽车」),模型直接输出文字或图片结果,不需要为每个任务单独设计模型结构。SenseNova-Vision 在检测、OCR、关键点、分割、深度估计等十几个任务上,性能追平甚至超过专用模型。它不是你明天就能用的产品,但指明了一条路:未来一个通用模型就能看懂世界,不用再拼积木。

📄 原文摘要(英文)

We formulate computer vision as unified multimodal generation, where heterogeneous visual tasks are expressed in the native text and image generation spaces of a unified multimodal model, without task-specific architectures. Under this formulation, SenseNova-Vision uses natural-language instructions and optional visual prompts to specify tasks, target regions or views, and decoding conventions, and generates responses as text for symbolic outputs, images for dense spatial predictions, or mixed text-and-image outputs for compositional tasks. To support large-scale training, we convert diverse computer vision annotations into instruction-response examples compatible with these generation spaces, resulting in the SenseNova-Vision Corpus, a computer-vision instruction-response corpus spanning text, image, and mixed targets. Starting from an off-the-shelf pretrained unified multimodal model, SenseNova-Vision is trained primarily on this corpus, with auxiliary multimodal data used as a capability-preserving mixture, and requires no task-specific prediction heads or architectural modifications. The resulting model covers a broad range of vision tasks, including detection, OCR, keypoint estimation, segmentation, depth estimation, surface normal prediction, point maps, and camera pose estimation, while supporting language-defined variants that combine category, color, region, and other visual cues. Experiments show that a single unified model can match leading task-specialized systems across structured visual understanding, dense geometric prediction, segmentation, and multi-view visual geometry. These results suggest unified multimodal generation as a scalable route for integrating computer vision capabilities into general-purpose foundation models. The model and corpus are publicly available.

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部