小模型靠“分科训练”打赢7倍大的对手
一家企业把200多个内部应用的大模型流量，硬生生合并到一个模型上，而且这个模型比原来的主力小7倍，却在自家评测里全面反超。做法不是把目标揉在一起练，而是拆成三个“专科”:指令跟随、函数调用、内部任务，各训一个专家，再用SLERP把三个专家“平均”成一个。每个专家暴露的毛病都不一样——语义崩塌、过度调用、废话太多——得分别对症下药。它不是你明天能用上的东西，但给了一个反直觉的信号：模型不是越大越好，把训练目标拆开、再合并，可能比一味堆参数更划算。
📄 原文摘要(英文)
Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7times larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.