On March 25, 2024, Anthropic published its policy position arguing that effective, independent third‑party testing is a necessary part of governing large, frontier generative AI models. The company says creating a testing regime with input from industry, government, and academia is one of the best ways to reduce deliberate or accidental societal harms from AI.
Why a third‑party testing regime is needed
Anthropic contends that today’s frontier models — examples include Anthropic’s Claude, OpenAI’s ChatGPT, and Google’s Gemini — do not fit neatly into sector‑specific regulatory frameworks because they are general‑purpose and can be adapted to many downstream uses. Downstream systems inherit both capabilities and weaknesses from the frontier models they rely on. Given their power and broad applicability, these systems present risks of serious misuse or accidents, so independent validation and oversight are required.
The company also points out that frontier systems can exhibit emergent, autonomous behaviours that are hard to predict or measure (for example, inserting vulnerabilities into generated code or taking actions that contradict human intentions during multi‑step tasks). Developing tools and tests for these behaviours now is insurance against such behaviours manifesting in widely deployed systems.
What a robust testing regime should look like
Anthropic outlines several design principles for an effective third‑party testing regime:
- increase trust in AI systems among people and institutions;
- be narrowly scoped so compliance is not so burdensome that small companies are disadvantaged;
- apply only to the most computationally intensive, large‑scale systems so the majority of AI products remain out of scope;
- enable cross‑country coordination through shared standards and experiments with mutual recognition agreements.
Key ingredients are effective, widely trusted tests to measure behaviour and misuse potential, plus legitimate third parties that can administer tests and audit company testing procedures.
Practical elements and a staged process
Anthropic recommends a two‑stage testing model: a very fast, automated first stage that covers a wide range of risks and is biased toward avoiding false negatives; and, if issues are flagged, a more thorough second stage using expert human‑led elicitation. They also advise an initial practice period in which companies run pilot tests (sometimes with third‑party oversight) so that tests can be validated and shown to be feasible.
Anthropic highlights national security as a clear area where third‑party testing is especially useful. The company suggests identifying AI capabilities that, if misused, could threaten national security — for instance, significantly speeding up the design of biological threats or enabling complex cyberattacks — and developing targeted evaluations for those capabilities. If a model shows such capabilities, deployment policies should be adjusted (for example by removing certain capabilities from broadly deployed versions or gating capabilities behind ‘‘know your customer’’ checks), and relevant government agencies should be informed.
Who would perform the testing?
Anthropic expects a diverse ecosystem of testers:
- private firms: companies may subcontract tests to specialized firms (Anthropic cites working with Gryphon Scientific) or perform legally mandated tests for others similar to private audits;
- universities: academic researchers who currently have subsidized or free access to models could run their own evaluation initiatives, potentially supervised or elevated by government bodies;
- governments: some tests, particularly those concerning classified national security capabilities, may be mandated by law and executed by government agencies.
Because general‑purpose AI is a new domain, Anthropic says the structure of this ecosystem is unclear now and will emerge through experimentation, which is why work should start immediately.
Anthropic’s contributions and recommended government actions
Anthropic describes steps it will take to support the development of fair testing regimes, including:
- prototyping testing regimes through its Responsible Scaling Policy (RSP) and sharing lessons learned;
- testing third‑party assessments of its systems with contractors and government partners;
- deepening frontier red‑teaming to better identify risks and mitigations;
- advocating for government funding of institutions that develop AI evaluations (for example, in the United States: NIST, the US AI Safety Institute, the National AI Research Resource, and access to Department of Energy supercomputers for AI testing);
- encouraging governments to build national research clouds (such as the National AI Research Resource) so academia and government can independently study and test frontier AI systems and develop evaluation science.
How testing relates to other AI policy concerns
Anthropic returns to two recurring policy topics: openly disseminated or open‑source models, and regulatory capture. The company notes that openness has driven scientific progress in AI, and that broadly many current models are safe to disseminate. However, if future models become capable of catastrophic misuse, the norms around open dissemination may need to change.
Anthropic suggests safety guarantees for released systems (for example, classifiers that detect and block misuse attempts or gating fine‑tuning behind know‑your‑customer rules plus contractual obligations). The company emphasizes it is not an impartial arbiter on openness, so broadly accepted third‑party tests are needed to define unacceptable misuses and to evaluate both controlled (API) and openly released (weights) models.
On regulatory capture, Anthropic argues that third‑party testing capacity can reduce the chance that well‑resourced actors shape regulation to their advantage. Conversely, industry‑led approaches with high compliance costs could favour large firms and disadvantage smaller players.
A cautious approach to advocacy
Anthropic stresses that regulations impose administrative burdens on governments and regulated parties, and tend to be hard to reverse once enacted. Therefore, it advocates for ‘‘minimal viable’’ regulation that is practical to enforce and feasible to comply with, while remaining open to feedback.
Conclusion
Anthropic concludes that building a third‑party testing ecosystem is central to ensuring the safety of frontier AI systems. The company sets out practical proposals — from fast automated screens to expert human evaluations — and commits to supporting governments, academia, and industry in prototyping and funding the institutions and infrastructure needed to create broadly trusted, effective testing regimes that both reduce harm and preserve competitive innovation.
