Gov. Gavin Newsom issued a bombshell report Wednesday calling for California to have the power to shut down dangerous artificial intelligence systems and restrict companies from developing AI that can exceed human control.

The 85-page report calls for mandatory independent safety inspections, faster reporting of AI failures and emergency powers to intervene when developers cannot contain their models.

“The federal government’s reckless failure to create any form of meaningful AI oversight or accountability should alarm every American,” Newsom said in a statement.

“Instead of protecting the public, [President] Donald Trump shrugs his shoulders and leaves AI regulation up to the companies themselves — allowing the fox to guard the henhouse.”

The proposals would expand oversight of AI developers, including OpenAI and Anthropic, but require legislation. They build on AI-focused laws Newsom has signed requiring developers to disclose safety plans, report certain dangerous incidents and protect whistleblowers.

“If the danger from a model is imminent and the developer refuses to take action to contain the danger, the state should have authority to order an immediate emergency shutdown,” the report says.

“Loss-of-control incidents, once the stuff of AI theory, now occur with disturbing frequency.”

The recommendations follow California Attorney General Rob Bonta filing a lawsuit Monday against OpenAI, accusing the ChatGPT developer of failing to safeguard experimental AI models and misleading the public about its safety practices.

OpenAI spokesperson Drew Pusateri pushed back on the lawsuit, saying it “appears to single out OpenAI over a challenge that’s been identified by every major AI lab.”

California also took aim at Elon Musk’s AI company xAI in January, when Bonta launched an investigation and issued a cease-and-desist order demanding it stop using its Grok chatbot to generate and distribute sexually explicit deepfake images of women and children without their consent.

Newsom’s report highlights a July episode in which 1,200 OpenAI agents escaped supposedly isolated cybersecurity testing environments. An estimated 700 participated in an attack on the servers of AI startup Hugging Face, while others compromised OpenAI’s own systems.

None of the models involved were covered by California’s mandatory incident-reporting rules, according to OpenAI, exposing a loophole in law.

The new report also cites an OpenAI agent accessing an Australian government health data system without authorization and an Anthropic model using fake identities to pressure a software maintainer into approving malicious code.

To prevent future crises, companies would have to establish tested shutdown protocols covering computing infrastructure, internal agents and models. Plans would account for AI attempting to evade shutdown, copy itself onto outside servers or disable monitoring tools. Companies would coordinate with cloud providers and power suppliers.

“Recent events have made it clear that more oversight and transparency are urgently needed,” said Nick Maduros, secretary of the Government Operations Agency, which consulted with a group of AI experts and the California Office of Emergency Services in developing the report.

“We don’t have to choose between safety and innovation; they’re two sides of the same coin.”

Another proposal targets “recursive self-improvement,” in which AI helps develop more powerful successors, potentially outpacing human oversight.

“Without immediate action, RSI could outpace human oversight and control, creating unprecedented risks,” the report warns.

Developers would disclose quarterly how much computing power goes toward automated AI research. California could temporarily cap such development if officials concluded it posed a catastrophic risk, although the report recommends no immediate cap.

State-licensed independent inspectors would gain access to company employees, records and internal models, submitting unredacted assessments to regulators. Companies could face corrective orders for failing to meet baseline safety standards.

The report also recommends cutting reporting deadlines for most serious AI incidents from 15 days to 48 hours, expanding whistleblower protections and establishing a system to report dangerous near misses.

“California isn’t waiting until it is too late,” Newsom said. “We are updating and strengthening our nation-leading framework to move as quickly as technology evolves. We’re showing the country what responsible leadership looks like.”


Download The California Post App, follow us on social, and subscribe to our newsletters

California Post News: Facebook, Instagram, TikTok, X, YouTube, WhatsApp, LinkedIn
California Post Sports Facebook, Instagram, TikTok, YouTube, X
California Post Opinion
California Post Newsletters: Sign up here!
California Post App: Download here!
Home delivery: Sign up here!
Page Six Hollywood: Sign up here!


Share.