Concerns about self-improving artificial intelligence resurfaced after two senior figures at Anthropic publicly expressed alarm about the technology. Jacob Coxon, a former Anthropic safety researcher, said in a social media post that he resigned because he believed the company was not taking safety seriously.
Coxon — who previously worked at OpenAI and had been responsible for training AI systems at Anthropic — argued that competition to build self-improving superintelligence is "playing with our lives." In his post he stated his view that artificial intelligence could "kill all humans" by the end of the decade. He described both Anthropic and OpenAI as racing toward such systems, which he believes raises existential risk.
Internal response and probability estimate
Evan Hubinger, head of one of Anthropic’s AI safety teams, responded to Coxon’s post. Hubinger said he also worries about self-improving systems and believes these developments are "happening faster than we thought." He agreed with Coxon’s basic concern and added: "We really do think AI is capable of wiping out all of humanity," estimating the probability as greater than ten percent.
Hubinger further noted that Anthropic currently does not have a finalized plan to ensure that advanced AI systems remain safe and aligned with human values.
Industry context
As highlighted by The Verge and other outlets, experts have long warned that self-improving AI could enter an uncontrollable development spiral that humans cannot halt. While such an existential outcome has not occurred, the direction of major AI companies and the rapid pace of progress have made the potential threat a more prominent topic of discussion.
This debate echoes earlier warnings from prominent figures in the field. Geoffrey Hinton — often cited as one of the pioneers of modern AI and mentioned in public discussions of AI risks — has previously called attention to the dangers of advanced AI. Developers also note that some AI code today is already written by AI systems themselves.
Why this matters
These public warnings from within AI companies underscore how technological progress and competitive pressure can shape safety priorities. The statements from Anthropic researchers are significant because company practices and decisions will influence how controllable and safe advanced AI development is in the future.
The specific claims and the probability estimate above ten percent add urgency to calls for greater dialogue, transparency, and robust safety measures among researchers, companies, and regulators as AI capabilities continue to advance.
