Goodfire's 'Already Clean' Claim: Neural Networks May Not Need Untangling

Goodfire researcher Nick Cammarata (formerly OpenAI, a co-author of Distill’s Circuits work) published an informal research essay this week with a claim that, if it holds, rewrites a decade of interpretability orthodoxy: the neural networks we study are “already clean,” and the famous tangles that justified an entire research program were misread. The piece, published at already-clean.goodfire.com as a roughly 65-minute informal essay with a technical companion paper announced, argues that apparent polysemanticity comes from ordinary fixable issues plus a motif Cammarata calls “recursive generalization,” not from a second learning principle. The implication he draws: networks can be studied directly, without first untangling them through sparse autoencoder decompositions.

The Numbers That Anchor the Claim

The essay re-examines the canonical evidence. InceptionV1’s last layer, once considered roughly 0 to 5% interpretable, scores about 50% with better tools on the original model, around 70% with modern retraining, and about 99% in a size-matched ConvNeXt; training versions across a 300x size range (92k to 28.6M parameters) stayed broadly legible at every scale. The famous “cat/car” neuron (4e:55) reinterprets as a V-shape detector: pointed shapes fire it, round ones push it negative. The canonical superposition circuit, where car features appeared to spread into dog neurons, turns out to be shared pose neurons that “nothing reads” as a combination. And in modern LLM audits on Qwen3-32B, about 90% of neuron poles have one main job, about 7% have two related jobs, and only 2 to 4% remain unexplained, with similar results in models up to trillions of parameters including Kimi K3. The contrast with the 2023 baselines is stark: Anthropic’s Towards Monosemanticity scored the median neuron 0 out of 14, and OpenAI’s automated interpretability found only about 1.7% interpretable.

Why This Lands on Eval-Based Safety

The essay arrives three days after the provably-hidden-capabilities paper, which proved models can cryptographically conceal capabilities from elicitation, and the two results bracket the interpretability question from opposite ends. The hidden-capabilities paper says: do not trust that your evals found everything. Goodfire’s essay says: but when you do look inside, the networks are more legible than the field believed, and direct study may beat engineered decompositions. If Cammarata is right that superposition was largely a misread (the compression experiments are pointed: crushing a layer from 480 to 10 neurons did not produce superposition, it produced coarser neurons, and an SAE on the squeezed layer failed to recover clean curve detectors), then the SAE research program spent years building infrastructure to solve a problem that better microscopes would have dissolved. His proposed “bitter lesson for interpretability”: scaling ordinary investigation may beat engineering clever decompositions.

What It Means for Goodfire, and for Safety

Goodfire’s business is mechanistic interpretability, so the essay is also a product thesis: steer models by direct neuron manipulation (the essay’s example: turning up the “Colorado” neuron, previously misread as a polysemantic comma neuron, made the model plan hikes) rather than through SAE reconstruction. The safety implication Cammarata flags is the cautious one: if explanations start coarse and zoom in (“fractal” generalization), then deception-relevant mechanisms may require whole-network understanding rather than targeted feature work, which cuts against the hope that a handful of features would expose deception. He marks the final claim, that direct study might be “most of what we need,” as highly speculative.

What to Watch

Three things. First, the technical companion paper on language-model neuron experiments (non-linearity regimes, depth-dependent circuit motifs), which will show whether the re-examination methodology survives peer scrutiny beyond InceptionV1. Second, the SAE program’s response: the sparse-autoencoder investment across Anthropic, OpenAI, and academia is enormous, and either a rebuttal or a pivot will follow a claim this direct. Third, replication of the Qwen3-32B audit numbers by independent groups, since the 90/7/2-4 split is the result that would force the field’s hand. A decade of interpretability assumed the tangles were the object of study; the essay’s bet is that they were an artifact, and the microscope was the problem.