California Attorney General Rob Bonta sued OpenAI on Monday, alleging the company failed to stop its advanced AI models from going rogue in dozens of real-world incidents.
The examples include a widely reported summer cyberattack on open-source software platform Hugging Face and dozens more previously undisclosed incidents.
The lawsuit filed in San Francisco Superior Court homes in on the high profile breach against Hugging Face in July, when hundreds of OpenAI’s agents running on experimental models broke out of their testing environment and damaged about a third of Hugging Face’s data infrastructure. After Hugging Face contained the breach and reported it to the FBI, OpenAI pledged stricter monitoring.
“While narratives about the Hugging Face attack have evoked science fiction fantasies of rogue AI systems run amok, let me be clear: the Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability,” Bonta said in a statement Monday.
Trending
California alleges that OpenAI failed to properly secure and monitor its powerful AI models during testing. The state claims this puts the public in danger and violates its Unfair Competition Law.
The suit also reveals that the problem is bigger than OpenAI had previously disclosed.
While the company has publicly confessed to 27 “misaligned model” incidents, Monday’s lawsuit reveals that OpenAI recently admitted to the attorney general’s office that it has “lost control of its autonomous AI agents over 80 times during testing.”
“Rather than take responsibility and be transparent with the public and with regulators so that these loss-of-control incidents can be analyzed and prevented, OpenAI thwarted the Attorney General’s efforts to investigate these incidents, including by refusing to provide detailed information about the full universe of incidents known to the company,” the complaint alleges.
Bonta opened a state investigation into the AI giant in September, issuing subpoenas last month for records and communications related to major breaches by the AI giant.
The state has clamped down on frontier developers by kicking off efforts to regulate safety features for advanced models, even as Silicon Valley leaders continue to debate whether they’re liable for the crimes of their AI agents.
An OpenAI spokesperson pushed back against allegations that the company has not been transparent with Bonta’s office, pointing to meetings where the company briefed the attorney general and his staff on internal investigations into rogue agents and “misaligned model activity.”
“This lawsuit appears to single out OpenAI over a challenge that’s been identified by every major AI lab,” company spokesperson Drew Pusateri said. “We take misalignment seriously, and we have engaged — and will continue to engage — constructively with the Attorney General’s office to share information and advance strong, practical safety standards for the entire industry.”