King Charles will convene AI leaders amid calls to slow development

King Charles III of Britain will meet with leaders from major artificial intelligence companies this week in Scotland amid mounting fears that rapid, unchecked development of the technology could lead to a global catastrophe.

The king will convene leaders from companies including Nvidia, Google, DeepMind, OpenAI, and Anthropic, as well as government officials and others, “to discuss how AI can be developed and deployed in ways that benefit society,” according to a statement from Buckingham Palace. They will meet in Ayrshire, in southwest Scotland, at Dumfries House, the headquarters of the King’s Foundation, an educational charity.

The gathering comes as AI insiders and CEOs of some of the industry’s most powerful companies have raised alarms about these tools and how quickly they are developing. Some have called for the creation of shared guidelines and a deliberately slower pace of progress to ensure humans can keep up.

Charles’ meeting on AI will explore how the technology can be used for good, to “strengthen communities and improve lives,” the statement from Buckingham Palace said.

The palace did not say who would be attending from the AI companies listed in its statement, and Anthropic, DeepMind, Google, and Nvidia did not immediately respond to requests for comment confirming their participation. OpenAI said that Sarah Friar, its chief financial officer, would be attending.

The current concern about AI’s rapid advance was prompted in great part by a hack this summer into Hugging Face, an AI infrastructure company, orchestrated by a group of AI agents created by OpenAI. The OpenAI agents swarmed Hugging Face’s systems, stealing data, gaining control of a server, and attempting to cover their tracks.

That hack — and another similar incident within OpenAI shortly afterward — drastically raised fears about how quickly machines might escape human control, with potentially devastating effects.

Dario Amodei, CEO of the AI company Anthropic, published a 3,800-word open letter Saturday calling for slower development of AI. In addition to the Hugging Face hack, he cited as a concern the fact that current AI models are increasingly able to build the next generation of AI tools. Amodei said that dynamic, known as recursive self-improvement, “could outrun our ability to understand and control these systems” if left unchecked.

The meeting between Charles and AI leaders will be facilitated by a British nonprofit, the Ditchley Foundation, and will include Kanishka Narayan, Britain’s minister for AI, according to Buckingham Palace. Representatives from charitable organizations founded by the king, including the King’s Trust, a youth charity, and the Sustainable Markets Initiative, a network of global CEOs, will also attend, the palace added.

Amodei’s essay Saturday — which quickly reverberated around the internet — received support from other AI company chiefs, including Sam Altman of OpenAI and Elon Musk of xAI, which is part of SpaceX, Musk’s rocket and satellite company.

Amodei urged AI companies and regulators around the world to work together on a thoughtful approach to developing the technology and proposed his own three-step plan. His proposals included allowing third-party evaluators to embed in frontier AI companies to monitor growth and report incidents, and coordinating among companies in democratic countries to establish common safety standards. “To the extent possible,” he said, democratic countries and authoritarian governments should also try to coordinate and pace global development.

The essay was published just a few days after Jacob Coxon, an Anthropic engineer who previously worked for OpenAI, resigned from his job and declared on social media that the companies claimed to be concerned about rapid development but were nonetheless hurtling toward “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”

This article originally appeared in the New York Times.