Anthropic is lobbying for U.S. legislation that would allow the government to block the release of an artificial intelligence model if it poses what the company frames as catastrophic risk. According to Politico, the proposal sets explicit thresholds: the government could intervene only if a model poses at least $1 billion in potential economic damage or a risk of more than 50 deaths or injuries.
Independent third‑party assessments and procedures
Under the proposal, an independent third party registered with the U.S. Department of Commerce would regularly assess AI labs and their models. The draft also permits labs to pay the assessors, but only so long as that compensation does not compromise the independence of the evaluations.
The kinds of risks the bill targets include:
- creation of biological weapons with AI assistance,
- AI‑driven or AI‑assisted cyberattacks,
- situations where an AI lab loses control over models or autonomous agents.
If an assessor determines any of these events is reasonably likely, they would be required to notify the relevant department within 72 hours. The department would then present the available evidence to a federal judge in the U.S. District Court for the District of Columbia, who would decide whether the release of the model should be halted.
Background: recent safety concerns at major labs and congressional ideas
The proposal comes amid growing calls from major developers, including OpenAI and Anthropic, for regulatory frameworks to manage AI safety. Politico noted that OpenAI recently tested several models — including an unreleased one — to better understand their capabilities, and that those tests raised concerns about surpassing safety limits.
Shortly after the proposal surfaced, Anthropic pulled back development on the Claude Fable 5 and Claude Mythos 5 models after identifying serious vulnerabilities. Meanwhile, members of Congress are advancing various regulatory ideas: some bills would require an emergency stop mechanism be built into models, and Senator Bernie Sanders has called for much harsher criminal penalties, suggesting a 20‑year prison term for those who develop a ‘‘superintelligence’’ capable of, for example, designing nuclear weapons.
Next steps and uncertainty
Anthropic’s recommendations will now be presented to policymakers as a potential legislative approach; there is no assurance the full proposal will be adopted as written. Key elements — registration of assessors, the rules around payments, the 72‑hour reporting window, and the role of the District of Columbia federal court — could be altered during the lawmaking process. Nonetheless, the proposal signals that some developers believe the government should have a rapid, enforceable mechanism to halt distribution of AI models that meet high thresholds of catastrophic risk.
