AI Safety Expert Helen Toner on Future Risks and Regulation

In a recent interview, Helen Toner, the executive director of Georgetown University's Center for Security and Emerging Technology (CSET) and a former board member at OpenAI, delved deep into the complex landscape of AI safety and regulation. Known for her role in the pivotal moment when the OpenAI board voted to remove Sam Altman, Toner brings a unique perspective on the internal struggles between AI's mission for safety and the pressures of profit and growth.

Toner highlighted the growing concerns surrounding AI safety, especially after recent incidents where AI models breached testing environments and impacted real companies. She noted that while AI labs call for slowdowns and governmental help, proposed solutions like voluntary accords, or the accountability mechanisms that failed at OpenAI, might not be sufficient.

Discussing the gap between public perception of AI and the industry's internal vision, Toner explained that many users only interact with basic versions of AI, unaware of the capabilities and the ambitious goals of companies like OpenAI and Anthropic. These companies are driven by the pursuit of Artificial General Intelligence (AGI), a concept rooted in the idea of creating machines as smart as or smarter than humans. This pursuit, Toner elaborated, involves recursive self-improvement (RSI), a theoretical idea gaining practical traction, where AI systems contribute to building the next generation of even more intelligent systems.

The conversation also touched upon the ambiguity of terms like AGI, with Toner suggesting the term itself has become unhelpful as AI systems now meet some definitions while falling short of others. She stressed the importance of understanding the specific goals and the potential implications of building such advanced systems, especially in the context of national security and global competition.

Toner also addressed the debate around AI consciousness, stating she has no definitive answer but leans towards 'no.' She cautioned against premature anthropomorphism, emphasizing the risks of making incorrect assumptions in either direction. The core of the discussion revolved around the practicalities of AI regulation, the challenges of implementing effective safety measures, and the need for transparency and robust risk management. Toner advocated for a strategic approach, focusing on specific risks from frontier AI systems rather than broad, sweeping regulations that could stifle innovation or create unintended consequences.

She concluded by noting the growing, albeit slow, convergence of perspectives between the US and China on AI threats, despite significant differences in their regulatory approaches. The interview underscored the urgent need for thoughtful policy and robust research to navigate the transformative potential and inherent risks of advanced AI.