把AI训练变成狼人杀：自己给自己出题打分
AI 变强靠的是「做对题给奖励」，但只有数学、代码这种有标准答案的题能这么练。写文章、聊天这种开放题，以前只能靠人打分或另一个 AI 当裁判，又慢又偏。这篇把开放题改造成「谁是卧底」游戏：几个 AI 拿到不同信息、完成同一个任务，然后互相投票找出卧底。因为卧底身份是事先定好的，投票结果就是现成的标准答案，不用人也不用裁判。实验里，这套方法在摘要、创意写作上超过了现有自改进方法，在数学推理上也有提升。它不是你明天能用上的，但思路很妙：把没答案的题，变成有答案的游戏。
📄 原文摘要(英文)
Reinforcement Learning with Verifiable Rewards (RLVR) has driven recent progress in reasoning-oriented large language models (LLMs) by enabling large-scale optimization. However, its applicability remains largely limited to domains such as mathematics and coding, where correctness can be deterministically verified. Open-ended tasks instead often rely on human preferences, reward models, or LLM-based judges, introducing evaluation bias, judge capability bottlenecks, and additional inference costs.Drawing on the principle of self-supervised learning, which constructs pretext tasks to derive supervision from the data itself, we propose Reinforcement Learning with Self-Verifiable Rewards (RLSVR), a task-transformation-based training paradigm for extending RLVR to open-ended tasks. RLSVR transforms open-ended tasks into verifiable proxy environments whose internal rules and interaction outcomes automatically generate reward signals. We instantiate RLSVR with SpyRL, a multi-agent self-play environment inspired by Who Is the Spy?. Agents receive asymmetric information, complete the same target task, and vote to identify a designated spy. Because the spy identity is predetermined, voting outcomes provide fully verifiable rewards, while successful identification remains closely related to output quality. Experiments on text summarization, creative writing, and mathematical reasoning show that SpyRL outperforms existing self-improvement methods on non-verifiable tasks and yields consistent gains on verifiable reasoning tasks. These results demonstrate that task transformation can extend scalable RLVR-based self-improvement beyond inherently verifiable domains. Models and code have been released at https://github.com/wangqinsi1/SpyRL.