AI审美不再打一个分，而是给一整套分布
现在的AI图像生成模型训练，靠的是给图片打分——但审美是主观的，一个分数根本说不清。这篇论文把打分从“一个数”变成了“一组概率分布”，比如一张图在“构图”“色彩”“真实感”等维度上各得多少分，而不是笼统的7.5分。他们用一个大模型（老师）先做复杂推理，判断出每张图在各个维度上的得分分布，再把这个能力压缩进一个小模型（学生），让小模型直接输出分布，不需要推理过程。结果：27B参数的老师模型在人类偏好匹配上达到89.6%，9B的学生模型也达到88.6%，远超传统方法。而且这个分布可以直接作为训练信号，让图像生成模型更懂人的偏好，比基线模型提升了41.3%的偏好率。它不是你明天就能用的工具，但告诉你一个趋势：AI审美正在从“一刀切”走向“多维度精细评估”。
📄 原文摘要(英文)
Reward models are central to text-to-image post-training, but visual preference is subjective and better represented as a distribution over rubric scores than as a deterministic scalar. Existing scalar, score-token, and pairwise reward models over-compress uncertainty and fine-grained score differences, while reasoning-based generative rewards provide stronger judgments but are costly to deploy and difficult to use as direct optimization signals. We propose Z-Reward, a teacher-student reward modeling framework that decouples reasoning-heavy judgment from efficient reward deployment. The teacher is a large VLM that uses reasoning to infer rubric-aligned score distributions, and is trained with Group-wise Direct Score Optimization (GDSO), which combines policy-gradient rewards from distribution expectations with direct pointwise and pairwise supervision on score distributions and score gaps. The student is trained with Reasoning-Internalized Score Distillation (RISD), which transfers the teacher's reasoning-conditioned score distribution into a compact VLM without requiring explicit reasoning chains at inference time. On our internally annotated evaluation set, the 27B GDSO teacher reaches 89.6% human preference accuracy, outperforming SFT, RewardDance, and GRPO, while the 9B RISD student reaches 88.6%, outperforming the OPD baseline and closely matching the larger teacher. We further show that Z-Reward can serve as a differentiable reward signal for text-to-image optimization, yielding a 41.3% net human-preference improvement over the SFT baseline.