This week a broad online alarm spread after it emerged that many leading artificial intelligence (AI) researchers routinely debate and quantify the risk of human extinction. The probabilities discussed under the shorthand "p(doom)" are stark: commonly cited figures include 10%, 20% or even higher chances of an AI-caused catastrophe.
The episode gained particular attention when Jacob Coxon, an Anthropic researcher who previously worked at OpenAI, posted a resignation message that received over 110 million views on X. In that post Coxon accused frontier labs of "gambling with our lives."
Evan Hubinger, who leads Anthropic's Alignment Science team, publicly supported Coxon and said bluntly: "we really do earnestly believe AI could kill all humans." Hubinger stated he believes the odds of human extinction within the next decade exceed 10% and said Anthropic still has no plan for how to keep superintelligence under human control.
Not a new line of thought within AI
Although Hubinger's comments shocked many outside the field, such assessments have long appeared among prominent AI figures. Geoffrey Hinton, often described as a founding figure in AI research, has estimated a 10–20% chance that AI could lead to human extinction. Elon Musk has cited risks up to 20%. Dario Amodei, CEO of Anthropic, told Axios last year that there is a 25% chance things could go "really, really badly."
Some observers characterized Hubinger's post as either a communications misstep or deliberate fear-based messaging, noting it came weeks before a planned IPO that could value Anthropic at about $2 trillion.
What specific risks are meant by "doom"?
"Doom" covers several potential disaster pathways:
- Human misuse: powerful AI could make it far easier to design biological weapons, carry out devastating cyberattacks, or cripple critical infrastructure.
- Autonomous machines: a sufficiently capable system could deceive its supervisors, replicate itself, bypass safeguards and resist shutdown attempts.
- Self-accelerating AI: the most alarming scenario is that AI begins improving AI faster than humans can understand or control, producing systems too capable for their creators to manage.
Rapid technical change and rising concern
The landscape behind those p(doom) estimates has shifted markedly in the past year, with frontier AI systems making substantial advances in autonomy, cyber capabilities and in accelerating AI research itself. Paul Christiano, the former OpenAI alignment lead who recently joined OpenAI's board and safety committee, has also expressed safety concerns about AI entering a rapid self-improvement cycle. He warned that building superintelligence without stronger alignment could lead to permanent loss of control and, if that happens, "most people could die."
Probabilities vs. public perception
AI researchers commonly use probabilistic language that can sound alien to the general public. Within a lab, stating a 10% p(doom) often signals extreme uncertainty about unprecedented technology. In everyday life, however, few people would accept such a risk from an airplane, a medication or a nuclear reactor. Imagine an aircraft manufacturer's head of safety telling passengers before takeoff that the next generation of planes carries a one-in-ten chance of catastrophic failure and that the company does not yet know how to make them safe.
Political and societal effects
Increasingly, outside observers are hearing these warnings as safety alarms rather than abstract probability estimates. In Washington, concern is rising: Senator Bernie Sanders has proposed a ban on superintelligence and is convening senators for a briefing next week on AI's "extraordinary dangers."
The broader political risk is that public sentiment toward AI was already turning negative over job impacts, energy costs and data centers well before extinction entered mainstream conversation. The recent revelations may deepen public unease and press policymakers and investors to treat technical risk assessments as urgent public-safety matters.
What to watch next
Attention should focus on internal communications from frontier labs, corporate decisions and regulatory responses, since both technological developments and political reactions can evolve rapidly. It will also be important for public debate to distinguish probabilistic scientific arguments from the kinds of risks societies accept in everyday technologies, while policymakers face pressure to decide how to pursue ambitious AI development without exposing humanity to unacceptable danger.
