训练视觉AI，数据混合比过滤更重要
训练视觉语言模型（比如能看懂图片并回答问题的AI）时，大家通常认为数据质量是关键，所以拼命过滤掉“脏数据”。但这篇研究用大规模实验告诉你：方向错了。他们搭建了一个系统性的数据实验平台，在160个数据集、6万亿token上测试了不同策略，发现真正提升模型能力的是“数据混合”——也就是把不同类型的训练数据（比如图片描述、图文交错文档、纯文本、指令微调数据）按特定比例搭配，而不是单纯过滤掉低质量数据。具体来说，指令类数据（教AI按指令做事）比图片描述类数据（只是描述图片内容）对模型提升更大，而且模型越大、训练数据越多，这种优势越明显。他们用这个发现训练出的8B参数模型，在33项核心任务上达到了63.6%的准确率，比当前最好的开源训练数据集高出5.4个百分点。这不是你明天就能用的技巧，但它给AI训练者指了一条更有效的路：别只盯着数据干净不干净，多想想怎么搭配不同类型的数据。
📄 原文摘要(英文)
Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies. We introduce DataComp for VLMs (DCVLM), a benchmark for controlled data-centric experiments to improve VLM training. As part of DCVLM, we collect 160 datasets spanning four data types -- image-caption pairs, multimodal interleaved documents, text-only, and instruction-tuning data -- into a corpus of 6T multimodal tokens. DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B-8B models and 6.25B-200B token budgets. Models are then evaluated on a carefully selected suite of up to 52 downstream benchmarks across 9 domains. We conduct extensive experiments on DCVLM and find that data mixing, not filtering, is key to a high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales. The resulting dataset, DCVLM-Baseline, enables training an 8B VLM to 63.6% accuracy on our 33-task core suite with 200B training tokens. Compared to FineVision, the state-of-the-art open VLM training dataset, this represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available at https://www.datacomp.ai/dcvlm/.