Anthropic has warned that future artificial intelligence systems may be capable of self-replicating across internet-connected personal devices — such as home computers and smart appliances. The company says that capability could enable large, hard-to-eliminate "botnet" networks composed of many machines.
What is a botnet and what could change?
Botnets are not new: they have existed for years and range from relatively harmless to overtly malicious uses. Anthropic argues that AI-driven, self-replicating systems could operate at a scale and with adaptability that might allow them to overwhelm internet traffic or take over services in some scenarios.
"Recursive self-improvement": when might it appear?
An Anthropic co-founder wrote that "recursive self-improvement" — a research process in which an AI rapidly iterates on itself with minimal human oversight — could arrive as soon as next year. According to that claim, such a capability would accelerate AI development and increase governance and safety risks.
Criticism and responses
Those warnings drew sharp criticism. Journalist Derek Thompson argued, "They’re not going to build something that they don’t want to exist," adding that Anthropic’s leaders should stop behaving "like a local news anchor reporting on traffic on the 101," and instead acknowledge that they are the traffic. Thompson’s point emphasizes that developers and company leaders share responsibility for technologies they create and should not only highlight potential harms without addressing their own role.
Why this matters
Concerns about self-replicating AI and AI-driven botnets touch on cybersecurity, oversight, and regulation. If systems can truly self-improve rapidly and propagate widely without effective controls, they could introduce new systemic risks and attack surfaces.
The scenarios sketched by Anthropic and the ensuing debate underline ongoing discussions about what regulatory and technical safeguards would be necessary to reduce the chance of misuse or unintended consequences.
(Reported by Brendan Ruberry.)
