Gemini Live’s New Video and Screen-Sharing Features
At the recent Mobile World Congress, I had the opportunity to sit down with Sameer Samat, the Head of Android, to discuss the latest updates to Gemini Live, a feature powered by DeepMind’s Project Astra. As we explored the new capabilities, I couldn’t help but reflect on the significant shift in the Android ecosystem’s focus, from personalized privacy controls and smart home integration to integrating AI-powered experiences into every aspect of the mobile experience.
Gemini Live’s New Video and Screen-Sharing Features
Gemini Live’s new features include the ability to point a smartphone camera at anything and immediately discuss it. I was taken to a special room in Android Avenue, where a ceramic vase sat on a table next to several glazed color samples. Gemini Live, in conversation, was able to help decide which colors would match best on the blank vase through voice queries along with live video from the camera on a Pixel 9 Pro XL.
Project Astra Comes to the Smartphone First
Crucially, these capabilities stem from DeepMind’s Project Astra, a research initiative to develop a "universal AI assistant" that simultaneously processes voice, visuals, and environmental context. Having experienced Project Astra firsthand at the Google campus last December, these new features immediately took me back to the power and possibilities I enjoyed in my demo.
Balancing Innovation and Consumer Trust
Our conversation pivoted to the contrast between the industry’s big bets on artificial intelligence and consumer readiness and acceptance of it. As I roamed the halls of MWC, I made note of the strategic importance of AI being signaled by almost every company on the floor. This contrasted with a general sentiment from attendees and consumers that these features and products aren’t nearly as desirable, useful, or secure as the industry wants them to believe.
Will Agents Eclipse Apps?
The rise of AI agents on smartphones, like Google’s Gemini 2.0, which powers features like Gemini Live and Project Astra, has raised concerns among some developers that their apps may be sidelined in the march toward agentic action. When asked how Samat squares up Google’s progress in agents with its long-standing relationship with third-party developers, he quickly admits, "I’m an optimist on this. I think it’s going to be a partnership where these things are going to work together."
The End Goal
As our discussion wound down, I asked Samat how he envisions the AI narrative evolving over the next few years. Samat made no bones about the importance of this moment in AI to Google and, more broadly, the entire industry. "AI is a super important technology," Samat said. "It’s as big a shift as mobile, as big a shift as personal computing."
Conclusion
In conclusion, the future of AI is bright, and Google is at the forefront of this revolution. With the launch of Gemini Live and Project Astra, the possibilities are endless, and it will be exciting to see how this technology evolves in the coming years.
Frequently Asked Questions
Q: What is the main focus of Google’s Android ecosystem?
A: The main focus of Google’s Android ecosystem has shifted from personalized privacy controls and smart home integration to integrating AI-powered experiences into every aspect of the mobile experience.
Q: What is Project Astra?
A: Project Astra is a research initiative to develop a "universal AI assistant" that simultaneously processes voice, visuals, and environmental context.
Q: How will AI agents impact the app ecosystem?
A: AI agents will not replace apps but rather enhance the user experience and provide more efficient ways of getting things done.
Q: What is the end goal of Google’s AI efforts?
A: The end goal of Google’s AI efforts is to make AI a seamless and integral part of our daily lives, helping us get more done and making our lives easier.
