“Love, Rendered” is a new short film about memory loss and storytelling. It shows how technology can help bring back memories that are fading. The story follows Burt and Ethelle Shatz, a couple married for more than 70 years, as they deal with Burt’s cognitive decline. One of the things he is losing is the memory of the day they met at a student co-op in Cleveland. That day was never photographed or filmed, so it only existed in their minds.
I took on the role of technical lead for the film. I knew the project would be as emotionally difficult as it was technically hard. Memory loss is personal to me. My grandfather had a stroke and lost his memory before he died. The last time I visited him, I was in my thirties. He was convinced I was still at university and was very happy that I was graduating. That bittersweet moment stayed with me.
That experience made me want to see if this technology could help me connect more deeply with my family through the same medium. When the project started, I asked my father for old family photos. I tested our image restoration on pictures from when my parents met and used our video models to animate them. Seeing my parents move as twenty-somethings showed clearly how these tools could help keep what matters most from slipping away.
“Love, Rendered” was directed by Academy Award-nominated filmmaker Liz Garbus. She produced it alongside Dan Cogan and Darren Aronofsky. It was created in collaboration between Google DeepMind and Primordial Soup, Aronofsky’s creative venture.
Exploring the mystery of memories
Liz and Darren both came to the film curious about the resilience of memory. While directing the documentary “Coma,” Liz saw fMRI scans light up when patients in minimally conscious states heard familiar voices or were shown images of loved ones. Years later, Darren encountered footage of a former ballerina with Alzheimer’s who, upon hearing “Swan Lake,” instinctively danced the choreography from her wheelchair.
These touchstones led the creative team to reminiscence therapy, the clinical practice of using sensory cues like songs, family stories, and old photographs to stimulate memories, spark conversations, and rekindle emotional connections. But what happens when a memory isn’t attached to such cues?
To recreate Burt and Ethelle’s unrecorded past, the filmmakers partnered with my team. Sitting alongside us, Ethelle became an active co-creator, correcting the curve of a staircase or the shape of a shoe heel.
Capturing human essence with AI
We worked alongside the team at Primordial Soup, using emerging AI models as an art medium and toolkit, guiding the technology with human direction every step of the way.
Bridging the gaps between the missing details required a two-part technical approach designed to preserve emotional truth.
- Image restoration: The team used generative models to restore black-and-white photos of Burt and Ethelle from their youth. The restored photographs helped to ensure that subsequent recreations remained faithful to their subjects.
- Pose and performance control: Engineers used our performance capture models to map Burt and Ethelle’s present-day micro-mannerisms — the specific tilt of Burt’s head, a brief hesitation in his speech pattern, the subtle crinkle around his eyes. Mapping those traits onto their younger likenesses brought the recreation to life.
Combining these two mediums allowed us to intertwine Burt and Ethelle’s past and their present. In doing so, we generated a “memory” that they told us felt authentic.
See my colleague Jess Gallegos explain the workflow in more detail.
As Darren noted during production, a tool — like a paintbrush or a hammer — does nothing until it’s guided by human hands. In “Love, Rendered,” machine learning was the tool we used to guide Burt and Ethelle back through time, helping them hold onto a vanishing moment.
You can watch the full film here to experience their love story, frame by frame.
Preserving your family’s story
Whether you’re looking to connect with a grandparent or preserve your family’s core memories, you can restore old photos in the Gemini app. Upload an image of the photo and ask Gemini: “Can you restore and colorize this photo? Preserve the appearance, expression, and pose of the people.”
