小模型干翻大块头：文档检索新思路
多模态模型通常又大又慢，因为它们是给「生成」设计的——先看再写。这篇反着来：只做「理解」，不做生成。研究者造了个 2.6 亿参数的双向编码器，把文字和图片像素直接塞进同一个 Transformer，不用单独接视觉模块。在文档检索基准上，它 2.6 亿的版本打败了所有 8 亿参数以下的模型，8 亿版本更是冲到 0.556 的精度。更狠的是，它把检索用的向量压缩了 255 倍，精度只掉不到 5%。这意味着以后企业做海量文档搜索，不用再租一堆显卡跑大模型，小模型又快又省。
📄 原文摘要(英文)
Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision encoders with causal language models. Visual document retrievers such as ColPali repurpose these models as encoders, carrying over the parameter and compute overhead of a VLM for a non-generative task. We introduce NeoMME, a family of 260M and 800M-parameter Multimodal and Multilingual bidirectional Encoders that process multilingual text and raw image patches in a single bidirectional Transformer encoder. Both models are pretrained from scratch with a masked discrete-diffusion text objective, conditioned on visible image patches for multimodal examples. Both support a 16,384-token context, enough to encode up to two standard 4K UHD images. To demonstrate its downstream capabilities, we fine-tune NeoMME with jointly trained dense and late-interaction heads. On the ViDoRe v3 benchmark, the resulting NeoMME-Retriever 260M outperforms all evaluated models strictly below 800M parameters with 0.523 nDCG@10, while NeoMME-Retriever 800M reaches 0.556. At a matched 2048x2048 image input size on an NVIDIA L40S, NeoMME-260M encodes pages with about 2x the throughput of ColModernVBERT. Hierarchical token pooling and asymmetric quantization compress late-interaction multimodal document embeddings by 255x while preserving over 95% of baseline nDCG@10. We contribute NeoMME to Hugging Face Transformers and release the pretrained backbone and retrieval-compatible checkpoints under Apache 2.0 at https://hf.co/collections/Hcompany/neomme.