Sparse Autoencoders, reading features out of superposition

A neuron in a language model is rarely about one thing: the same unit can fire for academic citations, Korean text and HTTP requests. The leading explanation is superposition, and the leading tool for working around it is the sparse autoencoder (SAE), an unsupervised model trained to rebuild a layer's activations from a much wider layer in which only a few units are active at once.

This is a reading note. It organises published work, and it reports no experiments of the studio's own. The one piece of hands-on work it touches is a fix to how SAELens loads JumpReLU SAEs, written up separately. The note follows the argument in four steps: why neurons cannot be read, what an SAE does about it, where the resulting dictionary goes wrong, and how an SAE is judged.

Feature

A direction in activation space that corresponds to something a human can name. Also called a latent when it is one entry of an SAE's dictionary; this note uses the two loosely.

Superposition

A model representing more features than it has neurons by storing them as overlapping, near-orthogonal directions.

Sparse autoencoder

A model that rebuilds a layer's activations from a much wider layer in which few units are active at once.

Dictionary

The set of directions the SAE's decoder can add together to rebuild an activation.

L0

The average number of latents active per token. Lower means sparser.

Why neurons cannot be read

Anthropic's Toy Models of Superposition (Elhage et al., 2022) trains a minimal ReLU model with n features and only d < n hidden dimensions. When features are dense, the model gives up: it represents only the few most important ones, each on its own orthogonal axis, and drops the rest. As features get sparser the trade flips, because two features that rarely fire together almost never collide.

In the toy model the switch between those regimes is a sharp phase change, not a gradual slide, and the superposed arrangements are regular: antipodal pairs, pentagons, tetrahedra and other shapes that keep features as far apart in angle as they can. Drag the slider below to cross the boundary, then fire features and watch the bars: each shows what a linear readout of that feature would report from the sum of the active ones.

Two cautions keep this from being over-read. First, polysemanticity is the observation and superposition is one explanation for it: a separate line of work shows polysemantic neurons can arise incidentally, from random initialisation and training dynamics, even when there are enough neurons to represent every feature (Lecomte et al. 2023). Second, the toy model is an existence proof, not a description of a frontier model; the qualitative sparsity trade is believed to carry over, and the clean polytopes have not been shown to.

There is also a structural reason neurons are the wrong unit. The residual stream is read and written by linear operations, so any rotation of its basis is equivalent and its individual coordinates mean nothing on their own. That is why a different basis, such as an SAE's dictionary, is needed there. And the word "superposition" has a second, unrelated use in work on continuous-latent reasoning, covered in R·019.

An overcomplete dictionary with few atoms awake

An SAE is a single-hidden-layer autoencoder trained to reconstruct an activation x from a hidden layer that is much wider than x but constrained to be sparse. The reconstruction is a sparse weighted sum over the columns of the decoder, which act as a dictionary of learned features.

x̂ = b_dec + Σᵢ fᵢ(x) · W_dec[:, i]        f = σ( W_enc (x − b_dec) + b_enc )

In words: the encoder decides which few dictionary directions are active and by how much, and the decoder adds those directions together to rebuild the activation.

The classic loss is L2 reconstruction error plus an L1 penalty on the feature activations. L1 induces sparsity but also shrinks every activation toward zero, a systematic underestimate. Newer architectures separate the question "is this feature present?" from "how strongly?":

Gated SAEs apply the penalty only to the selection step (Rajamanoharan et al. 2024). TopK keeps only the k largest activations, which controls sparsity directly and scaled to a 16-million-latent SAE on GPT-4 activations (Gao et al. 2024). JumpReLU replaces the ReLU with a thresholded activation and trains L0 directly with straight-through estimators (Rajamanoharan et al. 2024).

Features have no built-in labels. The standard move is to read each feature's maximum-activating examples, or to have a language model propose and be scored on a description. And the decomposition is causal, not only descriptive: clamping a feature changes behaviour as predicted, which is how the Golden Gate Bridge feature in Claude 3 Sonnet produced a model that steered every answer toward the bridge (Scaling Monosemanticity). How well that kind of steering works depends heavily on how the feature is selected, so it is a result about method, not a free property of SAEs. Clamping a feature is an intervention of the same family as the activation patching covered in R·026.

Where the dictionary goes wrong

Four failures recur, and they pull against each other: absorption gets worse as a dictionary gets wider and sparser, hedging gets worse as it gets narrower, splitting means there is no canonical dictionary at any width, and shrinkage is the sparsity penalty's underestimate. No single width or sparsity setting escapes all four. Shrinkage was covered in the previous section; the other three are below.

Absorption. Features form hierarchies: the token _short implies "starts with S". Firing both a parent latent and a child latent costs two active units; folding the parent's direction into the child's decoder and silencing the parent on that token costs one, with no loss in reconstruction. So the sparsity loss prefers it, and the parent feature "fails to fire where it should" (Chanin et al., NeurIPS 2025). The authors argue varying SAE size or sparsity is not enough to fix it.

Hedging. When an SAE is narrower than the number of true features and those features are correlated, reconstruction loss rewards mixing parts of the correlated features into one latent. Unlike absorption this is driven by reconstruction, not sparsity, and it gets worse as the SAE gets narrower (Chanin, Dulka and Garriga-Alonso 2025).

Splitting and composition. A coarse feature fractures into finer ones as the dictionary grows. Larger SAEs contain genuinely novel latents rather than just refinements, and meta-SAEs show many latents are compositions: an "Einstein" latent decomposes into scientist, Germany and famous person (Leask et al., ICLR 2025).

The fix has a cost. Matryoshka SAEs train nested prefixes of the dictionary to reconstruct on their own, so general features live in the early latents and cannot be absorbed by later specific ones. At L0 40 they reduce the absorption rate from 0.49 (BatchTopK) to 0.05, need about one latent per first-letter concept instead of three, and explain 70% of variance against 72% (Bussmann et al. 2025). The reconstruction gap is small but real, and because the inner nested dictionaries are narrow SAEs, hedging at those levels is a plausible cost (an inference from the design, not a measured result). It is a trade, not an escape.

Proxy metrics, and the sanity checks that followed

SAEBench (Karvonen et al., ICML 2025) compares more than 200 open SAEs across eight architectures on eight metrics, and its central finding is that gains on proxy metrics do not reliably translate into better practical performance. Matryoshka SAEs slightly underperform on the proxies yet substantially outperform on feature disentanglement.

Later work has pressed on whether the benchmarks themselves can be trusted:

Random models. SAEs trained on randomly initialised transformers often get auto-interpretability and reconstruction scores similar to trained ones, so a high aggregate score alone does not show that useful features were recovered (Heap et al.).

Random baselines. On synthetic data with known features, SAEs recovered 9% of the true features at 71% explained variance. On real activations, baselines with random directions matched trained SAEs on interpretability (0.87 versus 0.90), sparse probing (0.69 versus 0.72) and causal editing (0.73 versus 0.72) (Korznikov et al. 2026).

Probing. SAEs occasionally beat baselines on individual datasets, but no ensemble using SAEs consistently beat baseline-only ensembles, and simple baselines matched SAEs where SAEs first looked promising (Kantamneni et al. 2025).

The benchmark itself. An audit of SAEBench's metrics concludes that SCR and TPP "fail multiple lenses at their canonical settings and should not be used to evaluate SAEs", with sparse probing the most reliable metric it tested (Chanin 2026).

In short: an SAE hands you candidate features, not verified ones. The evidence that those candidates matter has to come from outside the reconstruction score: from random controls, from comparison against plain probes, and from causal tests.

Open questions

• QDo SAEs recover features of the model or features of the activations? A dictionary that compresses the data well need not correspond to the units the model's computation uses, which is part of why transcoders and weight-side decompositions exist.
• QWhat would a canonical unit of analysis even be? If larger SAEs hold novel latents and many latents are compositions, "the feature" depends on width, sparsity and seed, and the right size is a choice made for the task.
• QCan a metric tell a useful dictionary from a random one? Random baselines matching trained SAEs on several metrics means those metrics measure something other than what they were built to.