According to Reed, Dario Amodei and other leaders of frontier AI labs may genuinely care about AI safety, but calls to "pace the frontier" also align with their financial interests.
As Anthropic and OpenAI prepare to go public and try to persuade investors they can generate massive profits over time, a strategy built around spending billions to race toward ever larger and more capable models may not be the strongest sales pitch.
Even if large language models (LLMs) make only modest technical progress, the capabilities they already provide are sufficient to be extremely lucrative for these companies and for the wider economy. At the same time, increasing capability is becoming more expensive: it now takes exponentially more money to produce marginal gains that deliver only limited additional utility for most customers.
Solving a scientific milestone — as Reed puts it, a "Millennium Prize Problem" — can be valuable for prestige and for science, but it does not automatically translate into greater reliability for routine business tasks. Today, LLM reliability stems less from model size — bigger does not necessarily mean better and may never do so — and more from the surrounding software ecosystems.
The next major breakthrough may not come from larger models but from building AI systems that can continue to learn after deployment and run efficiently on ordinary hardware. Reed cites Richard Sutton's work at Oak Lab as an example: Sutton is attempting to develop a capable AI that could operate on about 20 watts of power (comparable to the human brain's energy use) and continuously update its weights, again mirroring aspects of biological brains.
AI-safety experts may be more concerned about these hypothetical future models than about today's large models because smaller, continuously learning systems would be more likely to leave the lab and be harder to switch off. Reed suggests this is a bridge humanity will have to cross eventually, but probably not in the immediate future.
More efficient AI would reduce costs for frontier labs. Yet that same efficiency could threaten the business models of companies such as Anthropic and OpenAI if customers could obtain comparable capabilities without relying on the powerful data centers those firms have invested in. Reed notes this could mean people run advanced AI locally instead of paying for expensive cloud plans.
Richard Sutton commented on a recent podcast: “You have to wonder about the large language models. They might be at risk when this eventually happens. I’m sure they’ll get a good run. They’ve already had a good run.”
It is worth noting that Jensen Huang, chief executive officer of Nvidia — the company that supplies critical hardware for AI development — disagrees with Dario Amodei and Sam Altman about the need to slow the pace of the frontier, CNBC reported.
Why this matters
The debate blends technological, safety and economic considerations. Beyond genuine safety concerns at leading labs, proposals to slow the frontier often reflect business incentives: model size and cloud-based business models are tightly linked. If AI shifts toward greater efficiency and continuous learning, it could change where and how AI is used, with direct consequences for firms that have made the largest infrastructure investments.
The issue is therefore not only how fast AI advances, but who pays for that progress and who benefits from it.
