He called for an AI slowdown, then told investors he’d race ahead

SAN FRANCISCO — After Anthropic chief executive Dario Amodei released a 3,800-word essay that called for the artificial intelligence industry to “slow down” to reduce the chance of disaster, two of his top rivals in the AI race quickly agreed.

“Dario is right,” Elon Musk, the CEO of SpaceX, responded in a post on X last month.

“I agree with Dario,” OpenAI CEO Sam Altman said in his own post soon afterward.

The unusual public compact between three of the world’s most powerful business leaders, agreeing to decelerate their companies to protect society from harm, resounded around the world. It followed public and political outcry over the potential dangers of AI, after an Anthropic employee resigned and said the industry was irresponsibly rushing to develop technology he said could wipe out humanity.

Six weeks later, there are few signs that the companies are relenting their pace. Advocates of more cautious AI development who cheered the calls for a slowdown are beginning to doubt it will actually happen.

Anthropic, OpenAI, Google and SpaceX have all launched new, more capable AI models since Amodei, Musk and Altman made their public stand. On Monday, Anthropic filed paperwork for what could be the largest initial public offering in history, telling investors that it was preparing to spend hundreds of billions of dollars to build more powerful versions of its chatbot Claude and other AI products in coming months and years.

“We anticipate making significant investments for the foreseeable future, including in compute and infrastructure, to ensure we have the technological resources to maintain and expand our position at the frontier of AI,” Anthropic wrote in its IPO filing.

AI companies should be judged on their actions, not on their statements, said Connor Leahy, the U.S. director of ControlAI, a nonprofit organization that has called for strict limits on AI development. “Would you believe this if an oil company said it?” Leahy said. “Watch the hands, not the mouth.”

A spokesperson for Anthropic did not return a request for comment. A spokesperson for OpenAI referred to an Oct. 23 blog post, in which the company said its work on containing and monitoring its technology was a “core part of responsibly pacing the development of our frontier models.” (The Washington Post has a content partnership with OpenAI.)

Some of the potential harms of AI cited by Amodei and his fellow chief executives in their recent calls for a slowdown have since become more concrete.

In the past month, cyberattacks apparently enabled by AI have hit companies around the world, including in South Korea, where an amateur hacker was able to break into multiple banks and steal customer information. Last week, OpenAI said it caught one of its AI agents actively seeking to evade a monitoring system deployed to make sure the agent didn’t go beyond its instructions.

An increasing share of the American public is concerned about the risks of pushing AI forward. A Washington Post-ABC News-Ipsos poll this month found that a majority of U.S. adults, 57 percent, said AI is a bad thing for society. YouGov found last month that half of American adults are somewhat or very concerned that AI could cause the end of the human race, up from 42 percent in late 2023.

Sen. Elizabeth Warren (D-Massachusetts) on Tuesday sent a letter to Amodei, asking him to reconcile his warnings about AI’s dangers with his drive to raise money on the stock market.

“I write with concern following your company’s disclosure that Anthropic’s advanced AI models could ‘pose catastrophic or existential risks to humanity’ at the same time that it seeks to register as a public company and sell shares in November,” the ranking Democrat on the Senate Banking Committee wrote in the letter.

OpenAI, Anthropic and other AI companies have taken some actions to impose safety measures. OpenAI and Anthropic have recently paused, at times, some of their work after models behaved in unexpected ways.

Anthropic has started to implement an idea referenced by Amodei in his call for a slowdown last month, inviting outside AI researchers into its offices to allow them to monitor development work and report on safety issues.

Amodei’s company has borne the brunt of recent criticism of the industry from the world of AI safety, a term used to refer to the community of researchers and nonprofit organizations that has spent years focused on the concern that AI could wipe out humanity and cause other harms.

Anthropic was founded by former employees of OpenAI — including Amodei — who said they left to build an AI company that took safety more seriously.

The AI safety community has generally been more optimistic about Anthropic’s approach than that of its rivals, said Malo Bourgon, chief executive of the Machine Intelligence Research Institute, a nonprofit organization that researches AI safety and advocates for strict controls on the technology.

The relationship between the company and the outside community of safety researchers is “slowly deteriorating,” Bourgon said.

“You’re not just a regular company,” Bourgon said of Anthropic, referring to the way its leaders talk about the gravity of the technology it is developing. “You want us to hold you to a higher standard and then you get frustrated when we do.”

After Anthropic lobbyists recently circulated a draft bill to regulate AI with members of Congress, AI safety advocates tore the bill apart on social media. The critics argued that the proposed law would preempt state laws that already constrain AI companies and said it did not match public statements from company leaders about their concerns about AI risks. The draft bill was first reported by Politico.

“This design makes perfect sense if you’re a company trying to minimize the risk that the government can overly constrain you,” Nathan Calvin, chief strategy officer at Encode AI, a nonprofit organization that advocates for AI regulation, wrote in a post on X directed at Amodei. “It makes little sense if you want American democracy to govern frontier AI effectively.”

Calvin was responding to a 2,000-word defense of the draft bill from Amodei, who said the company had made mistakes but aimed to make AI safer by staying ahead of all of its rivals.

“Our strategy has always been what I’ve called the ‘race to the top,’” Amodei wrote. “By showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry.”

The CEO also said he had made decisions that increased safety at significant cost to the company, including by getting into a conflict with the Defense Department by refusing to agree to allow their tech to be used for domestic surveillance or fully autonomous weapons.

AI companies have reported progress in addressing some safety issues. Earlier versions of AI chatbots were frequently sycophantic and encouraged delusional thinking among some users. The latest versions are less likely to encourage self-harm and will sometimes push back when a user appears to be at risk of engaging in harmful behavior, according to independent tests.

At the same time, new risks — like the recent cyberattacks — are becoming apparent as AI technology improves. Anthropic and OpenAI, widely seen as leading the AI race, have strong incentives to keep charging ahead.

Unlike established tech giants such as Google and Meta, the younger AI upstarts do not have profitable, established businesses that can be used to fund AI development.

OpenAI plans to follow Anthropic’s public offering with one of its own in coming months.