Over the weekend, four leading U.S. AI organizations — Anthropic, OpenAI, SpaceX and Google — signaled support for a possible pause in frontier AI development following a string of hacking incidents and striking demonstrations of new model capabilities. The proposal triggered a market dip on Monday as investors weighed what a slowdown would mean for economies that have relied on AI as one of the few dependable growth drivers this year.
That fragile unity unraveled quickly, however, as critics and stakeholders made clear they do not trust frontier AI companies or some of the safety advocates closely tied to them, making rapid oversight unlikely.
Why this matters
How, and how quickly, AI is regulated could reshape technological development and economic risk. Some political factions, business leaders and proponents of open models fear new rules would entrench the power of frontier AI firms. Others worry that calls for regulation may serve corporate interests rather than public safety.
Key factors behind the distrust
-
Money: Anthropic presents a paradox. Its leadership stresses safety to the point of arguing that its technology could pose existential risks, yet the company is widely reported to be preparing an IPO this week expected by many to target $100 billion in proceeds and a valuation around $2 trillion. That combination of apocalyptic warnings and large-scale fundraising deepens skepticism.
-
Safety ecosystem ties: For years, well-funded researchers and organizations have focused on AI safety. Critics argue some of this work is overly fixated on end-of-world scenarios and is too closely linked to frontier labs. Many safety groups employ former OpenAI and Anthropic staff, accept free model tokens for analysis, or take funding from organizations focused on existential risk. Anthropic has pointed to METR — which evaluated the HuggingFace incident at OpenAI — as a potential third-party assessor, but skeptics highlight financial and personal ties between the groups as disqualifying.
-
Business considerations: Several business leaders, including Nvidia CEO Jensen Huang and Microsoft CEO Satya Nadella, have backed open-weight AI systems that are cheaper and allow companies to build proprietary models using their own data. Most of the best open models were developed by Chinese startups, and some U.S. firms rely on them to cut costs or hedge against competition from frontier AI companies. If attempts to control AI led to bans or restrictions on Chinese models — given the expectation that China would not likely participate in a voluntary pause — many U.S. businesses might resist.
-
Politics: Senior administration officials, including White House AI advisor David Sacks, have dismissed regulation and oversight pushes as potential regulatory capture. A slowdown negotiated by frontier labs could preserve their lead, a concern voiced repeatedly by Sacks and venture capitalist Bill Gurley.
What to watch next
While the industry debates which evaluators are most independent and credible, some Washington-linked figures say they may form new organizations to fill an oversight vacuum and serve as third-party evaluators.
Bottom line
Getting Anthropic, OpenAI, SpaceX and Google to back a potential pause represented a meaningful step for proponents of tougher AI oversight. Yet deep distrust of the companies and of safety organizations with ties to them makes swift, broadly accepted regulation or self-regulation unlikely.
Maria Curi contributed reporting.
