A new study from South Korea’s KAIST and Naver AI Lab confirms that the reasoning steps a language model writes down match specific patterns inside its numerical states.
The researchers tested whether distinct actions like reading data, breaking down a problem, or running a calculation could be separated within the model itself. The signal is clearest in the middle layers of the network.
They defined eight recurring operations, including extraction, decomposition, formula recall, deduction, and computation. Three models—Qwen2.5-7B, Qwen3-8B, and Gemma4-31B—solved math problems. The team split the solution paths into segments and used GPT-5 to label each segment with one of those operations.
Reasoning steps are clearly separable inside the model
The different operations can be reliably told apart in the models’ internal representations. This holds across all three models tested. The separation peaks in the middle layers.
The researchers checked whether simple word choice could account for the effect. A classifier that only looked at the tokens used performed worse than one analysing internal representations. Position within the solution path did not explain it either. That means the internal states carry information about the type of reasoning step that goes beyond surface-level wording.
Same words, different representations depending on the reasoning step
Common function words like “a,” “is,” or “the” show up across very different reasoning steps. In the early layers, their representations are still jumbled together. By the middle and later layers they separate according to the surrounding operation. The same word gets a different internal representation depending on which reasoning step it belongs to.
The researchers also tested whether a reasoning step forms in isolation. When they blocked attention to the preceding 30 tokens through a targeted intervention, the signal for that operation weakened. Reasoning steps do not emerge on their own but build on the preceding context.
Even on incorrectly solved problems, the type of step the model was performing stayed identifiable. A flawed computation step still looked like a computation step internally, even when the result was wrong.
The separability held up in additional tests too. It replicated with Llama-3-8B. For Qwen3-8B the trained classifiers transferred successfully to GPQA-Diamond and MATH-500. The experiments are limited to math tasks and a handful of models. Whether these findings can be used to catch errors or steer a model mid-generation remains an open question for future work.
Safety and oversight
The relationship between text output and internal computation matters for AI safety. Reading the chain of thought is one of the few oversight tools available, according to OpenAI. Anthropic showed that models only disclose the hints they used in 25 to 39 percent of cases.
A method that translates a model’s internal vectors into readable text revealed that Claude Opus 4.6 processes more than what shows up in its output reasoning. With OpenAI’s Astra model, the Recurrent Depth technique shifts part of the reasoning into internal numerical representations, which is the space the KAIST study investigates.
What it means
For people building or auditing these systems, the finding suggests that what a model says is not always a complete map of what it is doing. If you rely solely on the text output to check for mistakes, you might miss internal errors that do not surface in the final answer.
