A new research paper argues that current world models fail because they ignore human beliefs. Systems like Sora, Genie 3, JEPA, and Marble track objects and motion but do not account for what people think, want, or consider socially appropriate. For service robots or medical assistants, this omission is a critical flaw because hidden mental states largely drive human behaviour.
The next state is also mental
The authors illustrate the gap with a simple example. If a cup is moved into a cabinet while a person is not looking, the scene appears correct to a purely physical world model. Yet the system predicts the wrong next action. Only a model that tracks the person’s belief about where the cup is can explain what they will actually do.
Their framework, called Mental World Modeling (MWM) and published on GitHub, extends classic world models with mental variables like beliefs, attention, goals, intentions, emotions, norms, and social relationships. The target agent only sees an egocentric partial view, while the world model holds the complete state.
Every action splits into a physical carrier, such as speaking, pointing, or grasping, and a mental payload, such as comforting, deceiving, or rejecting. The same gesture of sliding a cup across the table can be an apology, a deception, or an act of care. Only the world model holds the variables that tell them apart.
The authors explicitly do not claim to simulate consciousness. Mental states are hypotheses drawn from behaviour and context, not measurements. Systems built on the framework should represent uncertainty and keep their assumptions transparent.
MENTIS as a training-free reference implementation
To test the theory, the researchers built MENTIS, a modular pipeline that requires no additional training. It breaks the process into six steps. First, the scene is parsed and the ego perspective is rendered. Then the action options are split into physical and mental components, and the resulting states are simulated in parallel.
Each branch is then scored on three criteria: physical plausibility, mental consistency, and social appropriateness. After that, the pipeline makes a deterministic decision. Every stage writes a machine-readable intermediate result, so errors can be traced back to a specific step.
For evaluation, the authors built Menti-Bench, a dataset of 448 decision scenes: 320 text descriptions, 100 picture stories, and 28 sound-video clips. Each scene contains six response options and a human-created reference solution that documents not just the correct action but also the underlying mental and physical states. 78 percent of the scenes involve at least two characters.
More compute retries can’t replace mental modeling
The team tested eight language models, including five from OpenAI (GPT-5.6-Sol and GPT-4.1 among them) and three from Anthropic (Claude Fable 5, Claude Opus 4.8, Claude Haiku 4.5). The authors use the F1 score as their accuracy metric, combining precision and recall for the chosen action.
Across all models, the score climbs with each added modeling layer. Direct answers land at 63.3. Self-consistency, where the model answers the same question six times and picks the most common response, pushes that to 77.9. The full MWM pipeline reaches 87.9. Humans hit 98.5 under the same protocol.
The gains can’t simply be replicated by sampling the direct answer multiple times. The weakest model with MWM (GPT-4.1, 84.9) beats the strongest model using direct answers with self-consistency (GPT-5.6-Sol, 83.6). Additional tests confirm the framework’s core assumptions. Without the mental channel, models drop an average of 12.1 points. Without the physical channel, they drop 16.5 points. When both transitions are predicted independently instead of coupled, 6.4 points are lost.
Mental modeling has the biggest impact exactly where the theory predicts it should. In interpersonal scenes, the F1 score improves by 26.4 points. In object-focused scenes, the gain is only 14.0. Weaker base models benefit more from the explicit structure than stronger ones. The gap between MWM and direct answers is 28 points for GPT-4.1 but only 21 for GPT-5.6-Sol.
The bottleneck sits in the simulation step
To pinpoint where the remaining gap to human performance comes from, the authors replace individual pipeline stages with the human reference solution. The biggest single gain comes from perfect state transitions (+3.5 points), followed by perfect initial state (+2.8) and perfect observation (+1.7). When all intermediate steps are replaced with the reference solution, the pipeline hits 97 points.
About 80 percent of the remaining gap can be traced to prediction errors in the intermediate stages, mostly in the transition simulation. According to the authors, future improvements should start there. The challenge isn’t describing the current state; it’s simulating how the coupled physical-mental world changes.
A field that still hasn’t agreed on its own definition
World models are the big bet after pure language models. Demis Hassabis, who recently stepped down as operational head of Google Deepmind, has said he spends most of his research time on the topic and expects a “ChatGPT moment” for these systems. Investors are pouring hundreds of millions into startups like Odyssey. But what actually counts as a world model is still up for debate. An international team led by Peking University recently proposed a narrower definition that excludes text-to-video models like Sora because they lack feedback loops with the real world. Yann LeCun has argued for years that the generative approach is a dead end and instead backs abstract representations. The new paper groups Sora, Genie, and JEPA into the same family and criticizes all of them for the same omission.
The question of mental states loops back to a research area where language models have consistently struggled. A team from Meta’s FAIR lab and the universities of Washington and Carnegie Mellon already showed that models fail on demanding Theory of Mind tests, and they perform even worse at tracking world states than at attributing beliefs. The MWM framework applies these states from the outside through a preprocessing pipeline. Inside the models themselves, though, something similar is forming on its own. Anthropic found an internal scratchpad inside Claude that holds word-like thoughts that are never output, and without which multi-step reasoning breaks down.
What it means
For developers building agents that interact with people, the takeaway is clear. Adding a layer to track beliefs, goals, and social norms improves accuracy significantly more than simply increasing compute or retrying answers. The current standard of simulating only physical objects is insufficient for any task requiring social intelligence.
