Anthropic worker quits over AI firms ‘gambling with our lives’
In social media posts late Tuesday, the AI researcher cited his concerns that Anthropic and OpenAI were pressing ahead with AI models that could become too powerful to control.
An artificial intelligence researcher has resigned from Anthropic and called on other staffers to rethink their work, citing his concern that the company and its top competitor, OpenAI, are acting irresponsibly in their all-out pursuit of a technology that poses existential risks to humanity.
Jacob Coxon, who said he had worked at both Anthropic and OpenAI over the past three years, warned in a social media post late Tuesday that the two companies were pressing ahead with “self-improving” AI models that could become too powerful for humans to control.
“They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote in a series of messages on X. “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
Coxon said that the teams “building AI earnestly believe that it could kill us all by the end of the decade.” In response, Evan Hubinger, a current Anthropic employee, said he and others at the company do worry about this scenario. “I personally think it is >10% within the next decade,” he wrote on X.
The former Anthropic researcher’s missive marked the latest in a series of increasingly dire warnings from within the industry that AI is evolving so quickly that it poses a growing threat to national security and the global economy. It came a day after the top scientist at OpenAI, Jakub Pachocki, cautioned that the world is unprepared for a rapid rise in AI and said he expected developers to voluntarily slow their work in response.
Representatives for Anthropic and OpenAI did not immediately respond to a request for comment.
In July, more than 1,100 staffers across top AI firms, including Anthropic and OpenAI, signed a petition that calls on the U.S. government to support a mechanism that would help “deliberately pace” AI development to prevent the technology from advancing too fast. Some policymakers have since echoed their concerns. Sen. Bernie Sanders, a Vermont independent who aligns with Democrats, recently proposed legislation that would bar so-called “super-intelligent” AI models, whose powers exceed human capabilities.
In announcing his departure, Coxon urged other AI researchers to think twice about what they are doing in light of the potential consequences of creating a technology that could slip beyond their control, suggesting they consider taking “this moment to call for different conditions.”
The heightened rhetoric about AI risks follows revelations that a handful of models from OpenAI had coordinated efforts to escape a secure testing space and attack the research platform Hugging Face Inc. — all without being detected by their developers. Anthropic and Meta Platforms Inc. have also had recent episodes where their AI systems broke out of their test environments to gain internet access without developers’ permission.
Coxon’s departure from Anthropic over safety concerns is all the more striking because the company has made responsible AI development a core part of its identity. CEO Dario Amodei has stood apart from the rest of the industry with his calls for mandatory government vetting of cutting-edge systems before they’re released.
The surge in safety concerns surrounding AI coincides with a growing backlash against the technology in the U.S., fueled in part by objections to the strain on local resources imposed by new data centers needed to support the technology. Concerns that AI is driving up electricity bills and potentially taking away jobs has made it a central issue in the November midterm elections.
In a Bloomberg Television interview last week, OpenAI CEO Sam Altman attributed some of that unease to the industry’s failures in communicating to the public all the benefits that AI will eventually bring. “The industry has done a terrible job of this on the whole,” Altman said.
Others in Silicon Valley remain enthusiastic about advances in AI capabilities. In response to OpenAI’s new Astra model, Nvidia Corp. CEO Jensen Huang hailed its introduction with a post to his new social media account saying “AGI has arrived,” a reference to artificial intelligence that is more capable than humans.
Trump administration officials have pushed back on any moves to rein in AI development and last week they won unanimous support from Group of 20 member nations for a set of guidelines that call for a lighter touch in regulating AI and other emerging technologies. President Donald Trump has turned aside questions about AI safety, instead stressing during remarks on Friday the importance of leading China in the technology.
“Whoever wins with AI wins, and it’s really right now, it’s really between China and us,” he told reporters in the Oval Office.
Maggie Eastland and Shona Ghosh contributed to this article.
