On April 20, 2023, Anthropic published a policy proposal urging increased federal funding for the U.S. National Institute of Standards and Technology (NIST) to expand its work on measuring and standardizing artificial intelligence (AI) systems. The core argument is that accurate, widely accepted measurement tools are essential both to assess AI capabilities and risks and to underpin effective regulation.
Why measurement matters
Anthropic notes that AI systems have rapidly advanced in recent years, and AI-powered products are now widely deployed. Research has shown these systems can possess substantial capabilities as well as significant risks: some risks can appear abruptly as models scale, while others only surface after deployment. The field currently lacks broadly agreed methods to comprehensively measure and evaluate these risks, which complicates efforts to manage them.
NIST's existing role
The proposal points to NIST's long history in developing measurement science and technical standards. Examples of NIST's AI-related work cited include the Face Recognition Vendor Test and the AI Risk Management Framework. The widely used MNIST handwriting dataset, which helped progress in computer vision, was also originally based on NIST databases of handwritten characters and digits.
Although measurement work may seem technical or dry, Anthropic argues it is urgent: open-ended AI systems can behave unpredictably, and without robust measurement and test infrastructure policymakers and industry cannot reliably identify or mitigate harms before they reach the public.
Goals and expected effects of the investment
Anthropic proposes an ambitious funding program so NIST can further develop measurement techniques, standardize them across the field, and build community resources such as testbeds suitable for evaluating modern open-ended AI systems. The proposal lists several practical benefits:
- improving AI system safety through rigorous testing to detect and mitigate risks before public exposure;
- increasing public trust by providing independent, third-party validation of systems;
- giving government greater confidence when determining whether advanced systems are safe for general use;
- promoting innovation by creating incentives for developers to build better systems;
- enabling a market for certification and other positive incentives for participation.
The authors emphasize that measurement and standards are not a panacea for all AI risks, but they are a pragmatic, immediately actionable component of a broader "portfolio approach" to AI governance that includes internal controls, independent audits, and regulatory frameworks.
Funding shortfall and concrete ask
Anthropic observes that NIST's AI-related programs have been relatively under-resourced in recent years, a concern given rapid technological progress and widespread AI adoption. They recommend a concrete funding increase of $15 million over FY2023 levels (this figure includes NIST’s own requested $5 million increase). The goal is to ensure NIST has sufficient resources to meaningfully expand measurement and standards work.
Relation to other policy measures
The proposal frames additional NIST funding as complementary to other governance tools. Anthropic has previously advocated for a better-resourced NIST in testimony before the U.S. Senate Committee on Commerce, Science, & Transportation and in formal comments submitted to government requests for information. As Congress considered appropriations for FY2024, Anthropic urged lawmakers to consider this additional support.
Conclusion
Anthropic presents increased funding for NIST as a pragmatic, near-term policy lever to strengthen the measurement, testing, and standardization infrastructure needed to assess AI capabilities and risks. While not a standalone solution, the proposed investment is intended to support safer deployment and better-informed policymaking as AI systems continue to advance.
