搜图终于能说「要这件但换个颜色」了
搜图时你说「要这件衬衫,但换成粉色」——现有系统要么把「衬衫」「粉色」「忽略背景」全揉成一个黑盒向量,要么让AI自由发挥,结果它可能漏掉「粉色」或自己脑补出条纹。这篇把搜图变成了「查清单」:先把你的要求拆成六个固定格子(主体、属性、关系、要的、不要的、可忽略的),然后拿这张清单去核对每张图,符合的打勾、不符合的划掉,最后按得分排序。整个过程不用训练,直接跑。在五个搜图测试集上,它都赢了现有方法;还能把这套逻辑教给一个小模型,保留九成以上效果但成本低得多。它不是你明天就能用的产品,但方向很明确:以后搜图可以像填表一样精确,而不是靠猜。
📄 原文摘要(英文)
Real-world image search queries are multimodal and compositional: ``find this shirt in pink'' specifies an entity to retain, an attribute to modify, and context to ignore. Yet existing re-rankers either compress such multifaceted relevance into an opaque embedding or rely on free-form chain-of-thought that easily omits or hallucinates fine-grained constraints. Drawing on rubric- and checklist-based evaluation from NLP, we recast multimodal image re-ranking as a semantic constraint satisfaction problem and propose EviRank, which parses any query - text-only, image-only, or composed - into a unified evidence package: typed criteria across six semantic slots (e.g., entities, attributes, relations), each labelled required, forbidden, or ignorable. Re-ranking then reduces to evidence-conditioned verification, combining deterministic rubric scoring and evidence-grounded listwise comparison in a single training-free procedure. The explicit evidence can further serve as structured supervision for optionally distilling a lightweight student. Across five benchmarks spanning text-to-image, image-to-image, and composed image retrieval, EviRank achieves state-of-the-art performance, and the distilled student preserves over 90% of the teacher's capability at substantially lower cost.