Google’s Genie world model can now simulate real streets with Street View
We’ve all used Google Maps’ Street View to show a friend what our childhood home looked like or to drop that little person icon onto the streets of Paris. Imagine being able to do something similar, but in an even more immersive and interactive way—allowing you to really simulate the street and its surroundings, including adjusting the weather or seeing how it would look if there was a “Day After Tomorrow” scenario.
That’s one of the goals of Google’s latest integration. Starting today, Google DeepMind is connecting Street View data with Project Genie, the company’s general-purpose world model that can generate diverse and interactive environments. The new feature was unveiled during the Google I/O developer conference.
“It’s really powerful for both the agent use case and for humans to play with,” Jack Parker-Holder, a research scientist on DeepMind’s open-endedness team, told TechCrunch. “That’s always been the thesis of Genie.”
Parker-Holder gave an example: a new robot being deployed in London that rarely sees the sun. Genie could simulate those rare occasions when sunlight glints off Victorian housing, so the rays don’t shock the robot when they happen.
“You might say, ‘I’m going to New York City, but not this time of year,’” he continued. “‘It’s going to be snowy. I want to see what that block looks like in the snow.’”
Google has been collecting Street View data for 20 years via cars with cameras and individuals wearing “tracker backpacks.” The tech giant has amassed over 280 billion images across 110 countries and seven continents.
“With Street View, we have imagery from a large quantity of the world,” Jack said. “You can imagine how potentially powerful it is to combine this rich source of real-world information with an ability to simulate worlds.”
Last August, Google released its latest world model Genie 3 for research preview and opened up access to the tool to Google AI Ultra subscribers in the U.S., allowing customers to create interactive game worlds from text prompts or images. The goal is to use Genie for educational experiences, gaming, and robotics training.
Genie 3 is already helping one of Waymo’s simulators to train its self-driving cars on “exceedingly rare events” like tornadoes or a casual elephant encounter. Adding Street View data could help Waymo prepare to launch in more cities around the globe.
The difference with Genie, Parker-Holder said, is that those are all from the car’s point of view. Street View allows for simulating not only a world anchored to a real place but also shifting perspectives to other types of agents, like humans or robots.
Google is launching Street View in Genie to some Ultra users in the United States starting today, with access rolling out at scale over time. Global Ultra users will gain access over the next few weeks, per the company.
The researchers’ goal is to put this new capability into as many hands as possible, per Diego Rivas, a product manager at DeepMind. He cautioned that Street View and Genie in general are still experiments, so there’s much room for improvement in terms of accuracy.
In the samples Google showed me—such as an underwater simulation of a neighborhood I used to live in—the results are impressive and recognizable but still have video game quality rather than photorealistic. The models aren’t yet physics-aware; for example, in a snowy Joshua Tree scene, a woman running through cacti is not accounted for.
Compare that to Google’s image generator Nano Banana or its video generator Veo, both of which can now generate perfect text in infographics and understand cause-and-effect relationships like paper boats drifting on water. Physics isn’t hard-coded into these models but learned intuitively through passive observation, as a living being would.
“I think for this kind of model, it’s maybe six to 12 months behind video in terms of accuracy and quality,” Parker-Holder said.
Jonathan Herbert, director of Google Maps who started on the Street View team as an intern 12 years ago, thinks the real breakthrough is Genie’s spatial continuity. If you turn 360 degrees, the AI correctly remembers and simulates what was behind you. From there, it can build a new environment on top.
“We have long thought about how we could build out the best and richest model of the world on top of Street View data,” Herbert said. “It’s definitely been an idea of ours to use Maps Data in new ways for new kinds of AI research for quite a while.”
Key Takeaways
- Google DeepMind is integrating Street View data with its world model, Genie.
- This allows users to simulate real-world environments and test scenarios in an immersive way.
- The integration aims to improve the accuracy and realism of simulations for educational experiences, gaming, and robotics training.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
