科学想法也有基因组:AI能追踪思想的进化吗?
科学想法不是凭空冒出来的,它们像生物基因一样继承、变异、重组。研究者把每篇论文拆成一组最小、有证据支撑的“想法基因”,并记录它们如何从旧想法中继承、突变、丢失或引入新元素。他们构建了包含近2000条进化轨迹的基准测试,覆盖10个科学领域。结果:最强AI系统在追踪想法进化上的准确率仅27.3%,而且给AI提供完整的进化背景反而打乱了排名——不是所有AI都能用好这份“家谱”。这不是你明天能用的工具,但它揭示了一个前沿:AI要真正参与科学创新,必须先学会理解想法如何代代相传。
📄 原文摘要(英文)
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.