AI Pulse
📄 论文解读

同一个模型,教得更好就能学得更好

我们默认「老师做对了,学生照着学就行」。这篇发现:老师做对任务,不代表它做的过程值得学——成功轨迹里照样混着没根据的假设、绕圈子的重复、走过场的检查。他们的做法是给老师加一层「技能说明书」:用自然语言写明哪些坏习惯要压住、哪些好行为要多做,再让老师按这个去跑,学生只学跑出来的轨迹。同一个老师、同一批任务、同样的训练量,只改「怎么教」,4B 和 9B 的学生在终端操作、搜索、写代码三类任务上平均涨了 1.42 到 8.99 个百分点。更意外的是,训练时把技能说明一起留在数据里,评估时明明不给说明、监督信号也一模一样,成绩却普遍更高——说明「带着上下文学」比「只学答案」更能让行为长进去。它不是你明天能上手用的工具,但它改的是一个很基础的假设:模型蒸馏的瓶颈可能不在学生,在老师教的方式。

📄 原文摘要(英文)

Task success alone does not specify which behaviors post-training should reinforce. Even successful teacher trajectories can contain unsupported assumptions, unproductive repetition, or superficial verification. We propose Skill-FT (Skill-Guided Fine-Tuning), which uses natural-language skills to suppress undesirable habits and elicit underrepresented effective behaviors during teacher rollouts, then fine-tunes a target model on the resulting trajectories. We introduce AutoMaster, an autonomous post-training system in which an LLM agent develops these interventions through trajectory analysis, skill design, data selection, and feedback from rollouts and target-model evaluation. Across Terminal, Search, and software engineering tasks with 4B and 9B target models, AutoMaster improves mean performance over Vanilla SFT on successful unguided teacher trajectories by 1.42-8.99 percentage points, with the teacher, task pool, training set size, and SFT recipe held fixed. Together with the ablations and trajectory analyses, these results suggest that behavioral shaping can make a fixed model a more effective teacher. We further find that retaining skill instructions during SFT yields higher mean performance than removing them across all six settings, despite their absence at evaluation and identical supervised outputs. These findings suggest that preserving skill context as part of a training demonstration can support better behavioral internalization.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新