Google DeepMind Publishes Exhaustive Paper on AGI Safety

Google DeepMind on Wednesday published an exhaustive paper on its safety approach to AGI, roughly defined as AI that can accomplish any task a human can.

AGI Controversy

AGI is a bit of a controversial subject in the AI field, with naysayers suggesting that it’s little more than a pipe dream. Others, including major AI labs like Anthropic, warn that it’s around the corner, and could result in catastrophic harms if steps aren’t taken to implement appropriate safeguards.

DeepMind’s Predictions and Concerns

DeepMind’s 145-page document, which was co-authored by DeepMind co-founder Shane Legg, predicts that AGI could arrive by 2030, and that it may result in what the authors call “severe harm.” The paper doesn’t concretely define this, but gives the alarmist example of “existential risks” that “permanently destroy humanity.”

Exceptional AGI and Superintelligent AI

“[We anticipate] the development of an Exceptional AGI before the end of the current decade,” the authors wrote. “An Exceptional AGI is a system that has a capability matching at least 99th percentile of skilled adults on a wide range of non-physical tasks, including metacognitive tasks like learning new skills.”

Off the bat, the paper contrasts DeepMind’s treatment of AGI risk mitigation with Anthropic’s and OpenAI’s. Anthropic, it says, places less emphasis on “robust training, monitoring, and security,” while OpenAI is overly bullish on “automating” a form of AI safety research known as alignment research.

Recursive AI Improvement and AI Safety

The paper also casts doubt on the viability of superintelligent AI — AI that can perform jobs better than any human. (OpenAI recently claimed that it’s turning its aim from AGI to superintelligence.) Absent “significant architectural innovation,” the DeepMind authors aren’t convinced that superintelligent systems will emerge soon — if ever.

The paper does find it plausible, though, that current paradigms will enable “recursive AI improvement”: a positive feedback loop where AI conducts its own AI research to create more sophisticated AI systems. And this could be incredibly dangerous, assert the authors.

Proposed Solutions and Challenges

At a high level, the paper proposes and advocates for the development of techniques to block bad actors’ access to hypothetical AGI, improve the understanding of AI systems’ actions, and “harden” the environments in which AI can act. It acknowledges that many of the techniques are nascent and have “open research problems,” but cautions against ignoring the safety challenges possibly on the horizon.

“The transformative nature of AGI has the potential for both incredible benefits as well as severe harms,” the authors write. “As a result, to build AGI responsibly, it is critical for frontier AI developers to proactively plan to mitigate severe harms.”

Expert Disagreements

Some experts disagree with the paper’s premises, however.

Heidy Khlaaf, chief AI scientist at the nonprofit AI Now Institute, told TechCrunch that she thinks the concept of AGI is too ill-defined to be “rigorously evaluated scientifically.” Another AI researcher, Matthew Guzdial, an assistant professor at the University of Alberta, said that he doesn’t believe recursive AI improvement is realistic at present.

“[Recursive improvement] is the basis for the intelligence singularity arguments,” Guzdial told TechCrunch, “but we’ve never seen any evidence for it working.”

Conclusion

Comprehensive as it may be, DeepMind’s paper seems unlikely to settle the debates over just how realistic AGI is — and the areas of AI safety in most urgent need of attention.

FAQs

Q: What is AGI?

A: AGI is roughly defined as AI that can accomplish any task a human can.

Q: When can we expect AGI to arrive?

A: DeepMind predicts that AGI could arrive by 2030.

Q: What are the potential risks of AGI?

A: The authors of the paper warn of “severe harm” and even “existential risks” that could “permanently destroy humanity.”

Q: What is recursive AI improvement?

A: It is a positive feedback loop where AI conducts its own AI research to create more sophisticated AI systems.

Q: What is the proposed solution to mitigate AGI risks?

A: The paper proposes techniques to block bad actors’ access to hypothetical AGI, improve the understanding of AI systems’ actions, and “harden” the environments in which AI can act.

Smartphone Apps and Safer Driving

The Power of Feedback and Motivation

The AAA Foundation for Traffic Safety has published a study that highlights the effectiveness of using smartphone apps to track and improve driving behavior. The research found that drivers who received regular feedback on their driving habits, either through text messages or a data dashboard, showed significant improvements in their behavior.

Usage-Based Insurance and Smartphone Tracking

The study drew inspiration from usage-based insurance (UBI) programs, which use smartphone apps to monitor driving behavior and adjust insurance premiums accordingly. These programs typically offer discounts to drivers who demonstrate safe driving habits, such as avoiding hard braking, sudden acceleration, and speeding. However, many drivers are hesitant to participate in these programs, citing concerns about being tracked and judged.

The AAA Study

To overcome these concerns, the AAA researchers designed a study that involved 1,400 participants, who were divided into four groups:

• A control group, which did not receive any feedback
• A standard feedback group, which received weekly feedback on all monitored behaviors
• An assigned goal group, which received weekly feedback on a specific behavior chosen by the researchers
• A chosen goal group, which selected their own behavior to focus on

Results

The study showed that drivers who received feedback on their driving habits tended to improve their behavior. Specifically:

• 13% of participants reduced their speeding
• 21% reduced their hard braking
• 25% reduced their rapid acceleration

The Role of Feedback and Motivation

The researchers found that the key to improving driving behavior was not just the provision of feedback, but also the motivation to change. Participants reported that they were more likely to stick to safer habits if they were given a specific goal to work towards, and if they were able to track their progress.

Long-Term Effects

To determine whether the improvements in driving behavior were sustained over time, the researchers monitored the participants for an additional six weeks after the initial 12-week study. They found that the majority of participants continued to drive more safely, even without receiving feedback.

Conclusion

The AAA study suggests that smartphone apps can be a powerful tool for improving driving behavior and reducing the number of accidents on the road. By providing regular feedback and motivation, drivers can develop safer habits that can last a lifetime.

FAQs

Q: What is usage-based insurance (UBI)?

A: UBI is a type of insurance program that uses data from smartphone apps to monitor driving behavior and adjust premiums accordingly.

Q: What did the AAA study find about the effectiveness of smartphone apps in improving driving behavior?

A: The study found that drivers who received regular feedback on their driving habits showed significant improvements in their behavior, including reducing speeding, hard braking, and rapid acceleration.

Q: How did the researchers motivate participants to change their behavior?

A: The researchers found that participants were more likely to stick to safer habits if they were given a specific goal to work towards, and if they were able to track their progress.

Q: Were there any long-term effects of the study?

A: Yes, the researchers found that the majority of participants continued to drive more safely even after the initial 12-week study had ended.

Q: What does the study suggest about the potential of smartphone apps to improve driving behavior?

A: The study suggests that smartphone apps can be a powerful tool for improving driving behavior and reducing the number of accidents on the road.

OpenAI Convenes Experts to Inform Philanthropic Transition

As it prepares to transition from a nonprofit corporation to a for-profit, OpenAI says it’s convening a group of experts to “help OpenAI’s philanthropy understand the most urgent and intractable problems nonprofits face today.”

Expert Group to Focus on Urgent Problems

This group, which OpenAI says will incorporate feedback from “leaders and communities” in health, science, education, and public services, particularly within OpenAI’s home state of California, will be announced in April and submit insights to OpenAI’s board of directors in the next 90 days.

Board of Directors to Consider Insights

“[T]he Board will consider these insights in its ongoing work to evolve the OpenAI nonprofit well before the end of 2025,” OpenAI wrote in a blog post. “The Board recognizes the importance of engaging with the philanthropic community and those closest to the work to help inform how OpenAI’s philanthropy can best deploy its potentially historic resources.”

Background on OpenAI

OpenAI was founded in 2015 as a nonprofit research lab. But as its experiments became increasingly capital intensive, it created its current structure, taking on outside investments from VCs and companies, including Microsoft.

OpenAI today has a for-profit org controlled by a nonprofit, with a “capped profit” share for investors and employees. But as alluded to in the blog post, the company’s intention is to transition its existing for-profit into a traditional corporation, with ordinary shares of stock. The nonprofit would receive billions of dollars to cede control.

Stakes Are High

The stakes are high for OpenAI to complete the conversion expeditiously. If it isn’t successful by the end of the year, at least one of its backers, SoftBank, could claw back billions of dollars in pledged capital.

Conclusion

The expert group convened by OpenAI aims to inform the company’s philanthropic transition and ensure that it deploys its resources effectively. With the stakes high and a tight deadline looming, OpenAI must navigate this complex process carefully to achieve its goals.

FAQs

• What is OpenAI’s current structure? OpenAI has a for-profit org controlled by a nonprofit, with a “capped profit” share for investors and employees.
• What is the goal of the expert group? The group aims to help OpenAI’s philanthropy understand the most urgent and intractable problems nonprofits face today.
• When will the expert group submit its insights? The group will submit its insights to OpenAI’s board of directors in the next 90 days.
• What are the stakes if OpenAI fails to complete the conversion? If OpenAI isn’t successful by the end of the year, at least one of its backers, SoftBank, could claw back billions of dollars in pledged capital.