AI Pulse
📄 论文解读

一个模型搞定所有视觉任务,不用再换工具了

现在的AI做视觉任务,比如检测、分割、深度估计,通常每个任务需要一个专门的模型,换任务就得换工具。这篇论文把所有这些任务统一成一种“生成”问题:你只要用自然语言描述任务(比如“找出所有红色的车”),模型就能直接输出文字或图片结果,不需要为每个任务设计专门的架构。他们训练了一个叫SenseNova-Vision的模型,在检测、OCR、关键点、分割、深度估计等十几个任务上,性能都能和专门模型媲美。这意味着未来一个通用模型就能处理各种视觉需求,不用再为每个任务单独训练或部署模型。虽然它不是你明天就能直接用的产品,但它展示了AI视觉走向通用化的重要一步。

📄 原文摘要(英文)

We formulate computer vision as unified multimodal generation, where heterogeneous visual tasks are expressed in the native text and image generation spaces of a unified multimodal model, without task-specific architectures. Under this formulation, SenseNova-Vision uses natural-language instructions and optional visual prompts to specify tasks, target regions or views, and decoding conventions, and generates responses as text for symbolic outputs, images for dense spatial predictions, or mixed text-and-image outputs for compositional tasks. To support large-scale training, we convert diverse computer vision annotations into instruction-response examples compatible with these generation spaces, resulting in the SenseNova-Vision Corpus, a computer-vision instruction-response corpus spanning text, image, and mixed targets. Starting from an off-the-shelf pretrained unified multimodal model, SenseNova-Vision is trained primarily on this corpus, with auxiliary multimodal data used as a capability-preserving mixture, and requires no task-specific prediction heads or architectural modifications. The resulting model covers a broad range of vision tasks, including detection, OCR, keypoint estimation, segmentation, depth estimation, surface normal prediction, point maps, and camera pose estimation, while supporting language-defined variants that combine category, color, region, and other visual cues. Experiments show that a single unified model can match leading task-specialized systems across structured visual understanding, dense geometric prediction, segmentation, and multi-view visual geometry. These results suggest unified multimodal generation as a scalable route for integrating computer vision capabilities into general-purpose foundation models. The model and corpus are publicly available.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新