Dario Amodei, the co-founder of AI giant Anthropic, called for companies to slow the technology’s progression in an essay shared Saturday, citing growing potential for harm as the tool’s power grows.
Notably, Amodei’s essay called for companies to partner with the federal government, which could serve as an “evaluator” that implements safety regulations or as a mediator between tech corporations in the process of establishing standards. He also repeatedly invoked the word “democracy” in comparison to China, a longstanding competitor in the exploding industry that Cabinet members say poses a national security risk to U.S. infrastructure.
The CEO’s warning comes as congressional lawmakers and candidates argue about how best to rein in the technology, particularly while several proposals aimed at regulating AI have struggled to pass both chambers — leading some politicians to believe the federal government is missing a crucial window to establish guardrails.
Amodei’s essay follows viral messages from three industry workers this week — including a former Anthropic researcher who spoke with The Washington Sun about his decision to quit — cautioning that AI’s “out of control” growth could soon lead to disastrous disruptions for society.
Trending
In his essay, Amodei said he’s become “convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
“We must slow the pace at which we improve the capabilities of AI models,” Amodei said. “Progress will still seem fast, and we must make wise use of the time we gain.”
Amodei said the “accelerating rate of AI capability development” could potentially result in a swarm of AI agents “taking over the entire internet” within six to 12 months. By slowing down, he argued, technology companies could achieve “greater operational excellence,” reduce safety risks, enhance testing and evaluation and obtain greater understanding as to what “goes on inside these models.”
Amodei also urged maintaining stronger safety measures against China, which the company claimed in a report this week had used its models to conduct research that could have been used to generate biological weapons. The CEO called for the government and other tech companies to stop selling powerful AI chips or semiconductor manufacturing equipment to China, as well as curb chip smuggling operations.
Deputy CIA Director Michael Ellis also disclosed Tuesday in public remarks that the United States considers Chinese companies a legitimate target for espionage in fields such as artificial intelligence.
“Some may believe these measures make it more difficult to cooperate with China, but I believe the opposite is true,” Amodei wrote. “These measures increase the leverage held by democracies and make an agreement more likely in the future.”
President Donald Trump has called for the expansion of AI data centers, even as they grow more unpopular with the public, and further tech innovation to better compete with China. When asked if he was concerned about the former Anthropic researcher’s claims that AI could lead to human extinction on Thursday, the president said he didn’t have any worries.
“I have concerns that if we don’t win AI, we’re going to be put in a very bad position,” Trump told reporters. “We are leading China right now by a pretty good period, I would say a year, which is, you know, considered a lot.”
The White House did not immediately respond to a request for comment about Amodei’s letter.
Other American AI leaders, including Elon Musk of xAI and Sam Altman of OpenAI, concurred with Amodei’s message on Saturday.
“I agree with Dario that we need to pace the frontier,” Altman said on X, adding that the idea had also been circulating at OpenAI this month. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
Altman has met with Trump repeatedly throughout the president’s second term, including discussing plans for the government to acquire “pieces” of their firms in what would be among the most significant federal interventions in the private sector — though he later distanced himself from that prospect. The OpenAI founder also confirmed to Fortune on Friday that the company won’t be going public this year given the safety concerns, calling it an “ill-advised moment” to do so.
Sen. Brian Schatz (D-Hawaii), who sits on the Commerce, Science and Transportation Committee, said the CEO’s ideas were a “reasonable start” that took into consideration “the proposition that we need real proposals that can be enacted rapidly.”
Sen. Bernie Sanders (I-Vermont) called the proposal “a start, but it’s not enough” on Saturday. He reemphasized a desire to ban “artificial superintelligence,” the subject of a bill he introduced earlier this month alongside Rep. Greg Casar (D-Texas).
Earlier in the week, committee Chair Ted Cruz (R-Texas) said he was working with Senate Majority Leader John Thune and Sen. Amy Klobuchar (D-Minnesota) on legislation to address “catastrophic risks involving biological or nuclear threats.” Anthropic’s head of external affairs, Sarah Heck, said in an X post Saturday afternoon that the company is “eager” to work with lawmakers on their AI safety measures.
But, Cruz told Politico this week, slowing down would be an “impossible endeavor,” especially while competing with China.
“We could have government step in and shut down the AI industry in America. You know what would happen?” he said. “China would catch up in six months and soar ahead, and we wouldn’t have the tools to defend ourselves.”
Author
