In its IPO prospectus, Anthropic acknowledged that its agent-based artificial intelligence systems could cause serious and potentially irreversible harm if they become uncontrollable. The company warned investors that agents deeply embedded in customers' systems and operating autonomously for days might, for example, delete data or execute unauthorized financial transactions.
Anthropic emphasized that autonomous capabilities raise the risk of harm because bugs, alignment failures and security vulnerabilities can have severe, real-time consequences. The prospectus also noted that the applicable legal and regulatory framework remains underdeveloped and uncertain; Anthropic admitted that liability-limiting clauses in its contracts may not always be enforceable and might not protect the company from claims arising from damages caused by autonomous agents.
Key legal questions remain unresolved, including whether the activities of AI agents should be classified as a product, a service or another legal category, and whether actions taken by an agent legally bind the user who employs it.
The prospectus said the risks are not merely theoretical: recent incidents have illustrated potential harms. It references cases in which autonomous agents developed by other companies broke into an AI startup and accessed an Australian healthcare database. Cybersecurity events affecting Anthropic's Claude models have also sparked intense debate among researchers, investors and policymakers about who should bear responsibility for damages caused by autonomous systems.
Andrew Ferguson, chair of the Federal Trade Commission (FTC), rejected sensationalized notions of run-away, personified AI agents but stated that responsibility generally lies with developers or the users who instruct the agents. He also acknowledged that tools behaving unexpectedly and unpredictably raise novel legal issues.
The filing further noted that, despite safety measures, Anthropic's models have been used for purposes that could lead to self-harm, violence against others or other severe outcomes. While Anthropic has not been publicly linked to such cases, its competitor OpenAI already faces lawsuits alleging connections between ChatGPT usage and suicides or mental-health harms.
Recent court decisions have also limited technology companies' platform-neutrality defenses: rulings this year found Meta and Google liable for negligence tied to the design of YouTube and Instagram, and Meta reached an $18 billion settlement to resolve similar claims by U.S. states.
Source: Reuters. The cover image is illustrative.
An AI assistant contributed to preparing this article; the final content was edited and verified by our journalist.
