AI 教练自己出题：越学越难，越难越强
训练 AI 遵循指令时，通常用另一个 AI 当裁判打分。但裁判的评分标准是固定的，如果题目太简单，所有回答都差不多，裁判就分不出好坏，训练就卡住了。这篇论文让裁判同时当教练：它比较 AI 的两次回答，如果发现题目太简单，就自动给题目加一条新限制，比如从“写一封邮件”变成“写一封邮件，语气要正式，且包含三个要点”。这样题目难度会随着 AI 能力自动提升，始终让 AI 处于“刚好够不着”的挑战区。在三个复杂指令测试中，这种方法比固定题目或只改评分标准的效果都好。它不是你明天能用上的，但揭示了训练 AI 的一个关键思路：与其让 AI 适应题目，不如让题目适应 AI。
📄 原文摘要(英文)
Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals. While recent methods adapt these rubrics to the evolving policy during training, the training prompts themselves remain static, drawn from fixed corpora. This static approach often results in a critical misalignment between prompt difficulty and policy capability, leaving the judge unable to recover a discriminative reward signal when prompts fail to elicit quality variance among rollouts. To address this misalignment, we introduce LLM-as-a-Tutor, a framework that extends the LLM's role from judge to tutor: a single model serves as an examiner that pairwise-compares policy rollouts to detect non-challenging prompts, and as a generator that appends atomic constraints to them. This append-only design monotonically raises difficulty in step with the policy's capability, producing a self-calibrating training signal without external difficulty schedules. On three complex instruction-following benchmarks, our method consistently outperforms both policy-unaware baselines and prior policy-adaptive methods that adapt rubrics or rewrite prompts, suggesting prompt adaptation as a missing axis of policy-awareness in non-verifiable RL.