World models
Genie 4 adds sound and speech to generated worlds
Google DeepMind has announced Genie 4, a real-time world model that generates ambient audio and interactive dialogue alongside the scene, with Gemini steering events as you play. It is a research preview for trusted testers.
On 27 October, Google DeepMind announced Genie 4, the next version of its interactive world model. The headline change is that the model now generates sound as well as pictures: music, ambient noise and spoken dialogue are produced together with the world, in real time.
Genie 4 is a research preview for trusted testers. DeepMind says it will reach Google AI Ultra subscribers in Project Genie, and early access partners of the Gemini API and Google AI Studio, “soon”. There is a waitlist, but no public date.
How it works
DeepMind describes Genie 4 as building on the Gemini family. Gemini Omni anchors the simulation in real-world appearance and behaviour, while other Gemini models supply the reasoning that keeps a scene coherent over time.
Players use standard keyboard controls for movement and camera, and text prompts to change the world as it runs: the weather, the camera, or a new person or object. Gemini adapts the prompt to the surrounding scene. Between your inputs, it also watches the simulation and steers it so events keep progressing.
DeepMind highlights four areas: real-time shaping of the simulation, more realistic physics and human movement, native audio, and a planning loop. In that loop a Gemini model breaks a goal into steps, feeds them to Genie as text captions mid-simulation, and checks incoming frames to see whether each step worked. DeepMind shows the same setup with Gemini Robotics.
Sound and speech
Genie 4 is described as the first Genie model to generate audio natively. Each world carries the ambient sound of its setting. Characters also hold interactive dialogue that adapts to your actions and to events in the world, all generated live.
What is not published
The announcement is a showcase, not a technical report. It gives no parameter count, resolution, frame rate, latency or session length, and no benchmark scores. The demonstrations are recordings of real-time sessions chosen by DeepMind. Whether the dialogue stays consistent over a long session is untested in public, and so is memory of what happened earlier. DeepMind says it ran internal safety evaluations before the limited release.
Our take
Adding speech and sound moves world models from silent video towards something that behaves like a scene with people in it. For anyone building interactive worlds, the useful idea is the planning loop: a reasoning model directing a rendering model, rather than the renderer improvising alone.
That is the same split we describe in AI models as game generators: one layer keeps the story and rules, another performs them. Generated dialogue is only as good as the memory behind it, which we cover in memory in AI games. Until Genie 4 has public numbers, treat it as a direction, not a tool you can plan a project around.