A Look into the AI-Driven De-Aging Effect in Robert Zemeckis’ Movie Here
We mentioned a few months ago how convincing the AI de-aging of Tom Hanks looked in Robert Zemeckis’ new movie Here. Now we’ve learned more about how Miramax created the effect.
How Miramax Achieved the AI De-Aging Effect
Visual effects supervisor Kevin Baillie turned to the creative studio Metaphysic, which digitally augmented performances using its own generative artificial intelligence technology and team of digital character artists. And the process even allowed actors to rehearse while watching their young selves perform. The results could replace the Irishman in our pick of the best CGI movie moments.
(Image credit: Metaphysic / Miramax)
The Process of De-Aging
Baillie invited Metaphysic to screen test for the project back in 2022. For that, they recreated a young Tom Hanks, reminiscent of his appearance in Big while maintaining the emotional integrity of a contemporary performance by the actor.
Metaphysic’s proprietary process involves training a neural network model on a reference input, in this case video footage and images of a younger Hanks. A human artist then refines the results until the model is ready for production. From there, an actor or performer can drive the model, be it for a film or a live event.
Real-Time De-Aging
As Here moved into production, Metaphysic trained the initial neural network models to generate photoreal younger versions of the actors. Metaphysic VFX supervisor Jo Plaete and his team ran the workflow in real-time during the shoot, providing a visual reference of what the performers would look like.
This allowed Zemeckis to provide direction while viewing both the raw camera feed and the digitally augmented feed with the actors’ younger faces with only about a six-frame delay. Metaphysic also set up a camera and monitor system that allowed the actors to rehearse while seeing their younger selves. This ‘youth mirror’ system had just a two-frame delay and provided the actors with feedback that helped them to fine-tune their performances to better match their younger selves.
Conclusion
The AI de-aging effect in Here is a testament to the advancements in technology and the creativity of the team involved. The ability to recreate younger versions of the actors in real-time allowed for a more immersive experience and enabled the actors to refine their performances. The use of Metaphysic’s proprietary process and neural network models has opened up new possibilities for the film industry, and we can expect to see more impressive AI-driven de-aging effects in the future.
Frequently Asked Questions
Q: How did Metaphysic create the AI de-aging effect?
A: Metaphysic used its proprietary process involving training a neural network model on a reference input and refining the results until the model is ready for production. A human artist was then involved to fine-tune the results.
Q: How did the actors rehearse with the AI de-aging effect?
A: The actors rehearsed using a ‘youth mirror’ system that provided a visual reference of their younger selves with only a two-frame delay. This allowed them to fine-tune their performances to better match their younger selves.
Q: How many different ages were created for each actor?
A: Tom Hanks was portrayed at five different ages, Robin Wright at four ages, and Paul Bettany and Kelly Reilly at two ages each.
Q: What is the significance of the AI de-aging effect in the film industry?
A: The AI de-aging effect in Here is a testament to the advancements in technology and the creativity of the team involved. It has opened up new possibilities for the film industry, allowing for more immersive experiences and new storytelling opportunities.
