The last few days have crystallized a tension that's been building all year: the same AI labs racing to build ever more powerful systems are now publicly grappling with how—and whether—to slow down.

CEOs take AI risk to the UN Security Council

On September 23, OpenAI's Sam Altman and Anthropic's Dario Amodei addressed the UN Security Council directly, an extraordinary venue for a technology debate. Both executives pleaded with world leaders to come together to help manage the threats posed by rapid AI developments, with Amodei warning that AI could be a risk to humanity as a whole. Amodei didn't mince words, telling the council this is the most important global security issue facing the world today, and that differences between nations must be set aside to confront this threat.

Altman struck a more measured tone, warning against falling into "doomerism" and instead proposing a "middle path" built around human control, tangible benefits, and individual empowerment. He also pushed for concrete mechanisms, urging the creation of international standards for measuring capabilities, assessing risks, and preserving human oversight, plus faster reporting so the world can learn from failures before they become catastrophes.

A private safety regulator, built without Washington

While the CEOs were making headlines at the UN, their companies were already deep into a quieter project. Google, OpenAI, and Anthropic have spent months in working-group talks to create a shared oversight body, tentatively called the Frontier AI Standards Agency (or Standards Authority for Frontier AI, depending on the report). The plan traces back to a proposal Google DeepMind CEO Demis Hassabis made on July 14 for a FINRA-style, US-based standards body. The resulting organization would reportedly establish security and safety benchmarks starting in late 2026 or early 2027, set qualifications for model auditors, create incident-reporting rules, and support third-party pre-release testing. Notably, the three companies are launching it specifically to fill a regulatory vacuum after public-private governance efforts stalled, meaning it would operate entirely without government oversight.

Politics collides with the IPO calendar

The timing couldn't be more fraught for Anthropic, which is widely expected to pursue a blockbuster IPO. The day after Amodei's UN remarks, reports emerged that a Trump political adviser had circulated a White House memo casting him as the face of "AI doomerism." White House science adviser Michael Kratsios reportedly used the same UN session to dismiss international AI governance as a "globalist scheme." Investors are watching closely, with some already viewing the political fight as an added IPO risk factor.

Together, these threads show an industry trying to self-regulate in public while its own leadership becomes a political lightning rod—right as the biggest AI companies prepare to answer to public shareholders for the first time.

Sources

AI RegulationAI SafetyAnthropicArtificial IntelligenceDaily AI UpdateOpenAI