Concerns about artificial intelligence fall into two broad categories: dramatic, science‑fiction style fears about a future superintelligence that escapes control, and more immediate, evidence‑based risks arising from misuse, system unreliability, and the weaponization of AI. Current evidence and recent incidents make the latter group more pressing.
Why has alarm increased?
The pace of AI development has accelerated public attention. In 2012 AlexNet introduced a neural‑network architecture that helped spark the modern deep learning revolution; that approach now underpins many AI innovations, from smartphone face recognition to the generative models behind systems like ChatGPT. As capabilities have grown, so have reports of security incidents and potential misuse, which prompt comparisons to blackout or collapse scenarios.
Major AI actors — including OpenAI, Anthropic, Amazon Web Services and Microsoft — joined over a hundred other organizations in an open letter warning that there may be only months to prepare for a new wave of cyberattacks enabled by AI. At the same time, some researchers and former industry employees (for example Jacob Coxon) warn about existential risks tied to recursive self‑improvement and alignment failures; a few have estimated more than a 10 percent chance that AI could cause human extinction within the next decade. Those probability claims are highly contested, however: there is no agreed scientific framework to compute such odds, and public estimates vary widely. Jensen Huang, CEO of Nvidia, has put the chance of an apocalypse at 0 percent by 2030.
Concrete near‑term threats
Experts and company reports point to several realistic risks in the nearer term:
-
Biological misuse: Anthropic’s report warns that malicious actors have tried to use high‑performance models for designing rockets and bombs, creating lethal pathogens, and surveillance or influence operations; Anthropic describes five specific cases involving attempts to modify viruses or bacteria with the Claude model. Researchers at Stanford used AI to design 300 candidate bacteriophages and identified 16 that could infect E. coli — a demonstration of capability. Still, specialists stress that designing a pathogen is only one step: obtaining materials, equipment, specialized labs and expertise, plus highly automated laboratory systems, are required to turn a design into an operational weapon; such fully automated capacity does not currently exist.
-
Cyberattacks: AI already accelerates vulnerability discovery, code generation, and the creation of malicious tools. Much critical infrastructure relies on software that may not be designed to respond quickly to rapidly evolving attack techniques. Defenses will depend on hardening infrastructure, ensuring manual fallback options, and reducing unnecessary interconnections.
-
Disinformation: generative AI makes it cheaper and faster to produce believable fake images, audio and text. In January 2024, thousands of U.S. voters received robocalls falsely attributed to President Joe Biden; the calls used an AI‑generated voice and the creator was fined $6 million. The UN’s Global Communications Department has warned that uncontrolled AI use can amplify risks across the digital information ecosystem, and advertising funding can help spread deceptive content while undermining independent journalism.
-
Faulty AI‑generated intelligence: errors in AI‑produced analyses can have real geopolitical consequences. Earlier this year an AI‑generated report with incorrect information almost led the U.S. military to seize a Chinese ship, illustrating the danger of relying on unvetted AI outputs in critical decisions.
The politics and economics of apocalypse narratives
Some observers argue that apocalyptic language can serve PR and strategic interests. Critics claim that high‑profile warnings benefit incumbent firms and investors who might prefer regulation that entrenches established players. Steve Eisman and others suggest that promoting catastrophic scenarios can help shape future rules in ways favorable to large companies.
At the same time, people such as Jack Clark (Anthropic co‑founder) argue governments should act early — while the market has relatively few dominant players — to create rules that reduce accidents and prevent loss of control.
What should be done?
-
While most independent researchers assess the immediate probability of a runaway superintelligence as low based on current technology, AI’s rapid advance makes misuse and failure a real and urgent problem.
-
Risk reduction requires focused measures: strengthening critical infrastructure defenses, updating biosecurity regulations, improving tools against disinformation, and ensuring human oversight of AI systems used in high‑stakes contexts.
-
Transparency matters: because companies control what they disclose about capabilities and incidents, public assessment of risks is hampered. Better reporting and independent evaluation would improve understanding and policy responses.
In sum, apocalyptic visions are attention‑grabbing and sometimes play a political or commercial role in the broader discussion, but many researchers prioritize preventing and mitigating concrete near‑term harms — cyberattacks, biological misuse, unreliable AI outputs and the spread of AI‑generated disinformation — for which practical and regulatory responses exist.
Actors and sources referenced in this piece
OpenAI, Anthropic, Amazon Web Services, Microsoft, Nvidia, Jacob Coxon, Jensen Huang, Jacob Aron (New Scientist), Stanford researchers, Heidy Khlaaf (AI Now Institute), AI Futures Project, articles in Science and Nature, the UN Global Communications Department, and the 2012 AlexNet breakthrough.
