把可解释性写进训练，AI 反而更强了
我们一直以为，让 AI 可解释是给能力交税：先训一个黑箱，再事后想办法解释它。这篇直接把这个前提掀了——他们把「可解释」变成训练目标本身，和语言建模一起优化。结果出乎意料：模型越大，它的内部表示反而越清晰、越贴近人能懂的概念，而不是越乱。他们用这个思路训了个 8B 的模型，在只用别人 1/16 到 1/2 算力的情况下，能力依然能打。更关键的是，这个模型能让你直接看它生成某句话时到底用了哪些训练数据、哪些概念，还能不重训就改它的行为。这不是你明天能用的工具，但它指向一个更省力的方向：与其事后拆解黑箱，不如一开始就别造黑箱。
📄 原文摘要(英文)
Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.