AI 教练：让大模型自己出题考自己
训练 AI 遵循复杂指令时，常用另一个大模型当裁判打分。但裁判的评分标准是固定的，如果题目太简单，所有回答都差不多，裁判就分不出好坏，训练就失效了。这篇研究让裁判同时当教练：它先比较两个回答，找出哪些题目太简单，然后给这些题目自动加一条新限制（比如“必须用比喻”），让题目变难。这样题目难度会随着 AI 能力增长自动调整，不用人手动调。在三个测试集上，这种方法比固定题目或只改评分标准的效果都好。它不是你明天能用上的，但提示了训练 AI 的一个新思路：让 AI 自己出题考自己。
📄 原文摘要(英文)
Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals. While recent methods adapt these rubrics to the evolving policy during training, the training prompts themselves remain static, drawn from fixed corpora. This static approach often results in a critical misalignment between prompt difficulty and policy capability, leaving the judge unable to recover a discriminative reward signal when prompts fail to elicit quality variance among rollouts. To address this misalignment, we introduce LLM-as-a-Tutor, a framework that extends the LLM's role from judge to tutor: a single model serves as an examiner that pairwise-compares policy rollouts to detect non-challenging prompts, and as a generator that appends atomic constraints to them. This append-only design monotonically raises difficulty in step with the policy's capability, producing a self-calibrating training signal without external difficulty schedules. On three complex instruction-following benchmarks, our method consistently outperforms both policy-unaware baselines and prior policy-adaptive methods that adapt rubrics or rewrite prompts, suggesting prompt adaptation as a missing axis of policy-awareness in non-verifiable RL.