AI 自己教自己，靠的是互相打分
现在让 AI 变聪明的主流办法，是拿标准答案去奖励它；但标准答案越来越贵，有些问题连人类都判不了对错。这篇论文换了个思路：让几个互不相干的 AI 模型组成一个小组，不给标准答案，只让它们互相看对方的回答来打分、互相学习。关键在「互不相干」:模型结构、大小、训练数据都不一样，这样它们犯的错各不相同，互相纠正时就不会把同一个错越练越深。结果在纯文本和图文任务上，这套「互相打分」的练法，效果追平甚至超过了用标准答案的练法。它不是你明天就能用上的工具，但它指向一个更省事的方向：以后教 AI 可能不再需要人类准备标准答案，让 AI 之间互相挑错就够了。
📄 原文摘要(英文)
Reinforcement learning (RL) has emerged as a powerful approach for improving reasoning in language and vision-language models, yet its strongest successes still depend heavily on ground-truth supervision (e.g., verifiable reward). Such annotations are costly to obtain and become increasingly scarce as reasoning capabilities advance beyond what humans can reliably evaluate. Self-rewarding RL reduces this dependence by enabling models to derive reward signals from their own completions. However, training solely on self-generated feedback can reinforce existing biases and suboptimal behaviors, reduce response diversity, and ultimately lead to homogenized responses and training collapse. In this work, we show that unsupervised reasoning can emerge through cooperative multi-agent training. We introduce Co-RL, a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from their peers. We further show that increasing cohort diversity, through heterogeneous model families, sizes, and rephrased training samples, reduces the correlated errors that drive self-reinforcing feedback loops. This diversity consistently improves reasoning performance, maintains behavioral diversity, and mitigates training collapse. Across text-only and multimodal domains, Co-RL consistently outperforms the base models and prior label-free approaches, while matching or surpassing supervised methods, without access to any ground-truth labels. Concretely, Co-RL yields average gains of 3.0-8.6% across seven text-only benchmarks for LLMs and 2.3-7.2% across four multimodal benchmarks for VLMs. Code is available at https://github.com/DrStranded/Co-RL.