A researcher at Anthropic has resigned, warning that the rush toward self‑improving superintelligence is "gambling with our lives." The resignation was associated with Jacob Coxon, formerly of OpenAI. Coxon said Anthropic — believing itself to be the only responsible AI lab — was racing to reach such capabilities first, while OpenAI had not "internalized the civilizational stakes."
Timeline and risk estimates
Coxon argued that "things could be out of control" by the end of next year. Another Anthropic researcher expressed similarly grave concerns: "We really do earnestly believe AI could kill all humans," and said he personally assessed the probability of that outcome as greater than 10% within the next decade.
Executive‑level concern and proposals to slow progress
The alarm is not limited to departing researchers. Top executives at Anthropic, DeepMind, and OpenAI have signed a statement calling for government‑backed tools to slow AI development in case progress suddenly accelerates. The aim of the statement is to establish mechanisms that could decelerate development to reduce the risk of losing control and prevent severe harms.
Why this matters
At issue is that systems capable of self‑improvement could dramatically accelerate capability gains, and without adequate regulatory and technical safeguards, the consequences of losing control could be widespread and potentially catastrophic. The concerns voiced by researchers and company leaders underline that beyond technical innovation, governance and oversight of advanced AI development are urgent policy and industry priorities.
Next steps
Details and implementation of the government‑backed measures proposed by company leaders will require further consultation with regulators and stakeholders. Decisions by policymakers and practices adopted by developers over the coming months will be critical in shaping the trajectory of research and deployment of highly autonomous, self‑improving AI systems.
