把可解释性写进训练，AI 反而更强了
我们一直默认：AI 越强就越难懂，想让它透明就得牺牲能力。这篇直接把这个前提掀了——他们把「可解释」当成训练目标本身，和语言能力一起优化。结果在三个数量级的算力上，模型越大，内部表征反而越清晰、越接近人能理解的概念。他们用这套方法训出的 8B 模型，能力不输给用 2 到 16 倍算力训出来的开源模型，而且能直接定位输出是受哪些输入、哪些概念、哪些训练数据影响的，还能不重训就纠正行为。这不是你明天能用的工具，但它指向一个反常识的方向：透明不是能力的代价，可能是能力的一部分。
📄 原文摘要(英文)
Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.