语法不是单个神经元，而是一小撮神经元的合谋
我们总以为 AI 的「语法知识」藏在某个单独的开关里，找到它就能读懂模型。这篇用词性(名词、动词、形容词)当探针，发现真相更别扭：词性确实能被 AI 内部的活动精确还原，但没有任何一个「神经元」对应一个词性。它是一小组稀疏的神经元共同撑起来的，而且不同词性用的神经元组还不一样——开放类(名词、动词)和封闭类(介词、连词)的组织方式差别很大。这些小组在没见过的数据上依然稳定，说明不是死记硬背单词，而是真有一套结构。换句话说，AI 的语法是分布式合谋，不是原子零件。它不是你明天能用上的东西，但如果你关心「AI 到底怎么学会语言」，这是把钥匙。
📄 原文摘要(英文)
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use part-of-speech (PoS) categories as a controlled test case to study whether morpho-syntactic information is encoded by individual latents or by structured groups of features. We find that PoS distinctions are highly recoverable from SAE activations, but do not align with one-to-one latent / category mappings. This recoverability is not reducible to lexical memorisation, and Open and Closed PoS classes differ substantially. Categories are supported by compact groups of sparse latents, with substantial variation across tags. These groups remain stable on held-out data, while also showing overlap between related categories. Our results show that SAEs localise morpho-syntactic information in a distributed and category-dependent form rather than through atomic grammatical features.