Google Deepmind has released Genie 4, an interactive world model that calculates environments frame by frame in real time and, for the first time, directly generates matching background audio and character dialogue.
The system does not output finished videos. Instead, it renders a scene incrementally, reacting to input as it happens. Users navigate the space with the keyboard and steer events through text prompts. Gemini models operate in the background to plan what happens next, adapting the simulated world to user instructions and shifting conditions.
Audio and dialogue are now part of the render
Deepmind states this is the company’s first real-time world model to generate sound directly. Each environment produces its own background audio, such as wind moving through trees. The model also handles dialogue, with characters speaking to one another and reacting to user input or events in real time.
The company claims the biggest improvement in realism concerns how the model renders people. Characters move naturally and react to their surroundings, allowing users to control them directly.
Gemini models plan ahead to keep the world coherent
Genie 4 builds on the capabilities of the video model Gemini Omni to ensure the world looks and behaves realistically. Other Gemini models assist in planning the simulation to maintain consistency.
If a user changes the weather with a text prompt, Gemini adapts the instruction to the running scene. Deepmind says Gemini also steers events between inputs so the world develops in a way that makes sense.
Deepmind explains the process with multistep tasks. Gemini 3.8 Flash watches the scene continuously, breaks a goal into steps, and uses new frames to check whether each step worked. If the world changes or the user steps in, the model adjusts its plan.
Deepmind is also testing Gemini 4 Argon to tie its Genie work to its robotics research, Gemini Robotics. In one demonstration, Argon breaks the task of playing checkers into steps. Genie 4 receives these as text instructions and uses them to simulate Apptronik’s humanoid robot Apollo playing the game, with no physical robot involved.
Longer sessions still trip up the model
Deepmind calls Genie 4 an early, experimental model. The company notes the world stays consistent over short sessions, but visual details can shift and objects do not always persist in longer simulations or after several complex prompt changes.
Deepmind says it is still researching complex physical interactions, accurate real-time sound, and readable text.
For now, Genie 4 is only available to select testers as a research preview. It is set to reach Google AI Ultra subscribers in the Project Genie web app soon, along with partners who have early access to the Gemini API and Google AI Studio.
Google says all videos and audio tracks will carry its SynthID watermark, which invisibly marks AI-generated content.
Once they mature, world models could serve as training environments where robots and AI agents learn from simulated experience rather than just from data. Deepmind sees pairing Gemini with Genie as an early step toward hybrid world models that combine the reasoning of language and code models with pixel-level simulation.
