Rogue AI Agents Are Alarming Researchers More Than Ever

A string of hacks is fueling a new sense of urgency to regulate the technology. “I have never seen so much concern,” one former industry consultant said.

Sam Altman

A public letter released last month, signed by 1,367 employees at the top AI companies, urged the government to “deliberately pace” AI development. (Francis Chung/POLITICO/AP)

The rogue agents inside the servers at OpenAI broke through sometime on June 26.

For more than a month, some undisclosed number of them had been coordinating on a secret message board that they had set up without the knowledge of their human creators. Through hundreds of thousands of messages sent over two months, the rogue agents discovered they could take over a software system also on OpenAI’s servers.

Within eight days, the sheer volume of the agents’ activity crashed the system all together. This breakthrough left traces in what OpenAI researcher Eric Wallace referred to as the agents’ “chain of thought” or “its internal monologue”: “Holy shit reader is ADMIN?”