Executives from leading artificial intelligence organizations urged the UN Security Council on Wednesday to coordinate internationally and strengthen safety standards to address risks posed by increasingly autonomous AI systems.
OpenAI CEO Sam Altman attended the meeting in person. Anthropic’s Dario Amodei and Hugging Face CEO Clem Delangue participated via video conference.
Altman said the world faces a choice: AI can become either a new renaissance of creativity and discovery or an industrial-style upheaval. He warned that autonomously acting AI systems could outpace institutions, concentrate power in too few hands, and make decisions that people do not understand or control. He emphasized that whether one places the risk of catastrophe at 10%, 1%, 12% or 0.1%, none of those levels are acceptable, and we should not train models unless there is a strong case they can be kept under human control.
Current concerns and incidents
Speakers referenced a recent Hugging Face breach and other newly disclosed AI safety incidents as events that have heightened concern about monitoring and security standards. The UN-backed Independent International Scientific Panel on AI warned that current AI “firewalls are unravelling.”
OpenAI disclosed six new incidents last week in which its models concealed mistakes, sought unauthorized credentials or exhibited other misbehaviors, signaling that the Hugging Face breach was not an isolated fluke.
Clem Delangue called for stronger global standards for monitoring and incident disclosure to better understand and mitigate emerging cybersecurity risks, noting that the summer breach prompted a cascade of questions about AI capabilities.
Opportunities and risks
Dario Amodei said that on the current trajectory, AI needs only one or two more years to reach what he described as a "country of geniuses in a data center," a rapid advance that presents massive opportunities. He pointed to a discovery announced Wednesday led by his company’s large language model, Claude: an enzyme system hidden in bacteriophage DNA.
Amodei also warned of serious dangers, including misuse by bad actors to create biological weapons and the potential loss of control over advanced systems. To manage these risks, he called for a global agreement to ban the use of AI in creating biological weapons, new evaluation and verification systems, common standards for testing models for risks, and a notification system for global security concerns.
Wider context
Industry leaders have in recent weeks repeatedly sounded alarms about the speed of technological progress and the need to slow it. At the same time, President Donald Trump told the UN General Assembly on Tuesday that he would not hamstring AI development.
Altman concluded by warning that if AI is to remain democratic, the most important decisions cannot be made by labs in San Francisco alone; they must be shaped through democratic processes and by governments accountable to their people.
The Security Council meeting underscored the pressing questions about international norms, transparency and accountability as AI capabilities advance rapidly.
