AI记性有个选择性失明：只记住主流说法
我们总以为大模型知道的比别人多；这篇发现，它知道得多，但记住的窄。研究者给AI出了一种考试题：问的都是有争议的冷门事实——真相本来就存在两种以上的说法。最强模型能把两个版本都答出来的只有52.4%，剩下几乎全是只记得其中一个、把另一个直接抹掉。把模型做大、多给思考时间，能提高回忆率，但抹不掉这种选择性失明。为什么会这样？因为谁出现在AI训练素材里的次数多，谁就赢了——少数派说法在语料里曝光不够，就成了AI记性里的盲区。它不是你明天能用上的工具，但它指出了一件事：AI给出的“标准答案”，可能只是它见过最多的那个答案，不一定是唯一的答案。
📄 原文摘要(英文)
Factual question answering (QA) typically assumes a single canonical answer, obscuring whether large language models (LLMs) retain divergent accounts of long-tail facts. To address this gap, we introduce ElephantBench, a closed-book knowledge probe comprising 1,094 questions generated through an auditable graph-based pipeline. The pipeline retrieves related documents from a low-exposure web corpus, identifies naturally occurring disagreements, and converts them into multi-account QA records. Each answer is verified against the originating documents and authoritative public web sources and is then reviewed by human annotators. Across 32 models, even the strongest model recovers both accounts on only 52.4% of questions, while on nearly all remaining questions it recalls one account but omits the other. Scaling model size and inference-time reasoning improve recall but do not eliminate this incompleteness. Corpus analysis further shows that exposure imbalance favors the dominant account, whereas greater minority-side exposure is associated with more complete recall. These findings establish ElephantBench as a reproducible knowledge probe for diagnosing epistemic myopia in parametric memory. More broadly, our graph-based benchmark construction pipeline provides an efficient and scalable way to turn long-tail corpora into source-traceable knowledge probes, supporting efforts to evaluate and advance the epistemic rigour of next-generation LLMs. Code is available at https://github.com/Tencent/ElephantBench.