Nick Cammarata argues that large language models are far easier to understand than researchers previously believed, suggesting the field of mechanistic interpretability is closer to clarity than the tangled knot theory suggested.
Cammarata, a researcher at the interpretability lab Goodfire, published a lengthy essay titled “Neural Networks Are Already Clean” to reject the superposition hypothesis he helped shape years ago. The essay challenges the standard view that artificial neurons pack unrelated concepts into single units when the network runs out of space.
Mechanistic interpretability seeks to trace how a model arrives at its answers internally. Ideally, each of the millions of artificial neurons would represent a single, recognizable concept. In practice, many neurons seemed to fire on completely unrelated things, a phenomenon researchers call polysemanticity. One early and famous example is a neuron in the image model InceptionV1 that responded to both cat faces and the fronts of cars.
The standard explanation was superposition. The theory states that when a network runs out of neurons, it packs unrelated concepts into the same neurons because they rarely get in each other’s way. The result looks like a knot. That is why many labs train a second model, such as a sparse autoencoder, to pull the concepts apart. They then study that model instead of the original.
Cammarata played a major role in shaping this view. With Chris Olah, he co-wrote the paper “Zoom In”, which introduced the first superposition circuit. In 2023, he started an automated interpretability project at OpenAI. At the time, only about 1.7 percent of neurons counted as understandable, even by the most generous measure.
The evidence may have been misread
Cammarata now thinks the most famous examples were misinterpreted. He says the cat-car neuron is really a V-shape detector. Pointed shapes like fox snouts or flames activate it, while round shapes like the letter O push its activation negative. So it does not represent two completely unrelated concepts like cat faces and cars, which means it is not an example of superposition.
He also reinterprets the superposition circuit from “Zoom In”, where a car seemed to hide among dog neurons. In his view, it is ordinary generalisation, and the circuit simply groups cars and dogs by which way they face the camera. He blames the mess in InceptionV1’s late layers on an outdated architecture and weak analysis tools.
Cammarata explains the remaining ambiguity with a pattern he calls “recursive generalisation”. He says a neuron grows like a branch that sprouts side shoots. One neuron that recognised the letter sequence “iy” later responded to “diy” and eventually to “do it yourself”, which does not contain “iy” at all. Unlike superposition, these groupings can teach you something about the network.
For language models, he offers preliminary numbers. In Qwen3-32B, about 90 percent of neuron poles have one main job and about 7 percent have two related ones. Only 2 to 4 percent remain unexplained. Goodfire says it sees similar results in models with up to trillions of parameters. AI agents that analyse neurons in bulk did most of the work. Cammarata borrows from Rich Sutton’s “Bitter Lesson” and concludes that in interpretability, too, simple investigation at massive scale beats clever decomposition methods.
Cammarata does not introduce a fundamentally new method. He goes back to the field’s original approach of studying neurons directly, just with better tools, larger models, and automated agents.
Readable networks would be a big win for AI safety
If Cammarata is right, that would be good news, especially for AI safety. Right now, apart from monitoring chains of thought that keep getting less reliable, there are few ways to tell whether a model is deceiving people or behaving dangerously. Cammarata doubts that alignment, the work of steering AI toward human goals, can be solved without interpretability, because otherwise we do not know what we are building. Deception has no reason to hide from the model’s own circuits, he writes. If humans become just another “reader” of the network, they can see the deception too, and errors might be fixed in a targeted way.
Roon, an OpenAI employee who posts under that pseudonym on X and is an influential voice in AI circles, was thrilled. He says he is close friends with Cammarata and would never have ended up at OpenAI without him. The work “may be remembered as a major moment in scientific history”, Roon wrote. He added that the news is so good even Eliezer Yudkowsky, one of the most prominent voices warning of an AI catastrophe, will have to lower his estimated odds of doom. In Roon’s view, alignment could become a “normal science”.
Even Roon has some reservations. Others in the field still need to confirm the results and he also suspects that interpretability breakthroughs will push AI capabilities forward. The fact that they are coming from outside does not bode well for the big labs, OpenAI included.
Nanda sees a useful tool, not a breakthrough
Neel Nanda is far more measured. He is considered one of the founding figures of mechanistic interpretability and leads the interpretability team at Google DeepMind. Before that, he worked at Anthropic and as an independent researcher. His views have shifted, according to him: Since mid-2024, he has grown more pessimistic about ambitious interpretability and has leaned toward pragmatic approaches. His team deprioritised research on sparse autoencoders. They did not think the technique was useless, but it did not seem worth the effort the field was putting into it.
Cammarata’s work is interesting but does not really change the big picture, Nanda writes. Neuron-based analysis could help with practical questions, like why a model did something questionable, and it is much easier to apply to frontier models. But he does not buy the idea that interpretability is close to being fully cracked. He expects neuron-based analysis to have serious flaws and miss a lot as well.
Nanda still thinks superposition is real. He argues that language models store far more facts than they have neurons, and weak activations in particular likely hold much of that information. He doubts that findings from small image models carry over. He also thinks the essay defines monosemanticity too loosely but does give Cammarata credit for checking his old work with modern tools.
Part of the dispute comes down to definitions: Cammarata has no problem with models representing more concepts than they have dimensions, and Nanda sees him as agreeing on that point. What Cammarata disputes is that networks arbitrarily pack completely unrelated things together.
The technical companion paper has not been released yet. As an interpretability company, Goodfire also has an obvious stake in the thesis. If Cammarata’s reading holds up, he thinks automated researchers could do in months what took humans nearly a decade with image models. He writes that there may not be much time left for that.
What it means
For people building and using these systems, the shift changes the approach to debugging and safety. Instead of relying on complex mathematical tricks to untangle a model’s internal logic, teams might simply run larger automated analyses on the raw data. This makes it easier to see why a model made a specific choice or to spot deceptive behaviour without needing to train a secondary model first. However, the practical application remains limited to specific questions rather than providing a full map of how the system works.
