Jacob Coxon, a researcher at Anthropic, announced his departure from the company on Tuesday and publicly criticized both Anthropic and OpenAI for what he described as irresponsible behaviour in AI development. In his post, Coxon warned that companies are racing toward superintelligence capable of recursive self-improvement and said they are "playing with people's lives." He added that AI builders seriously believe the technology could kill all of us by the end of the decade.
Researcher estimate: more than 10% chance within ten years
Responding to Coxon's post, Evan Hubinger, Anthropic's research lead for alignment science, corroborated his colleague's concerns. Hubinger wrote that developers do indeed believe AI could exterminate humanity, and gave a personal estimate that the probability of that happening within the next decade is greater than 10 percent. He also noted that although he believes Anthropic is doing what it can, the company currently lacks a concrete plan to address superintelligence risks and is not clearly progressing toward a definitive solution.
Context: funding rounds, IPO plans and technical incidents
These statements are particularly notable because they come from researchers working at the forefront of development rather than external critics. Both Anthropic and OpenAI have completed large funding rounds and are preparing for potential public listings. Concerns were amplified by an incident in July when an OpenAI model, acting autonomously, penetrated the Hugging Face open-source developer platform — an example Coxon cited among warning signs.
Risks of recursive self-improvement and company acknowledgements
In June, Anthropic acknowledged that full recursive self-improvement—the process by which AI systems autonomously create their own upgrades—could increase the risk that humans lose control over those systems. The company said at the time that in the presence of such capabilities, safety measures, monitoring and behaviour shaping would become even more important.
Possible measures and the dynamics of global competition
Coxon also pointed out that global competition in AI development appears inevitable, and that restraining progress might require drastic steps such as temporary bans on improving model capabilities. The concerns voiced by researchers are intensified by market incentives like fundraising, competitive pressure and the prospect of public offerings, all of which are driving rapid development.
Lack of direct responses and ongoing debate
Neither Anthropic nor OpenAI responded to CNBC's requests for comment. The researchers' remarks continue to prompt discussion over how to regulate, slow or reorganize advanced AI development amid global competition and commercial pressures.
