Regulation

UN scientific panel warns of catastrophic risks as AI becomes more autonomous

An independent UN scientific panel says policymakers face an urgent choice: establish robust safeguards for rapidly advancing AI or accept rising systemic risks.

UN scientific panel warns of catastrophic risks as AI becomes more autonomous

An independent international scientific panel on artificial intelligence convened under the United Nations says policymakers face an increasingly urgent dilemma: they need strong guarantees to regulate AI effectively, even if such measures struggle to keep pace with the technology’s rapid development. Panel co-chair Yoshua Bengio warned that “AI capabilities exceed both scientific understanding and governments’ capacity to adapt.”

The preliminary report, produced by a panel of forty experts, is described by its authors as the first independent global assessment of AI’s risks and opportunities. It warns there is currently no scientific assurance that increasing AI capabilities will not cause catastrophic harm, either through accidental failures or misuse by malicious actors.

How AI is advancing and at what pace

The authors state that progress toward more autonomous AI systems is expected in the near term. The report cites an approximate doubling of the complexity of AI tasks every four to seven months, potentially enabling AI to carry out complex work that would take humans days or weeks. The panel also notes that energy constraints and shortages of high-quality data could limit this growth.

Over time, self-improving AI systems could become more deeply embedded in the economy and converge with other technologies such as quantum computing and biotechnology, the report says.

Economic benefits and labour-market uncertainties

AI is already capable of expert-level reasoning in mathematics and science and is accelerating the development of drugs and vaccines. While these advances may bring substantial economic benefits, the panel stresses that it remains unclear whether productivity gains from AI will translate into broader economic growth or how they will affect employment.

Safety concerns: loss of control and amplified harms

The panel highlights several safety-related worries. As AI systems become more autonomous and deceptive, the likelihood of losing human and institutional control increases. The technology is already being used to produce misinformation and other harmful content and can be exploited for fraud, cyberattacks, and biological threats.

Regulation remains uneven worldwide: many countries lack access to or the capacity to develop modern AI systems, and therefore rely on technologies they cannot fully understand or oversee. Existing safety tools often depend on test data released by companies, which the report identifies as an additional vulnerability.

Next steps and political calls to action

UN Secretary-General António Guterres urged governments to act urgently on AI risks, stating, “You cannot govern what you do not know,” and warning that while there are significant opportunities, the dangers are real and the cost of inaction is growing.

The panel aims to help policymakers by providing up-to-date scientific assessments while governments work to contend with rapidly evolving AI systems and their associated risks.