Researcher Toby Ord has modelled how recursive self-improvement (RSI) might play out in practice and argues that temporal and resource constraints will likely limit how rapidly an AI can escalate its own capabilities. As a result, an AI undergoing multiple rounds of self-improvement faces concrete bounds on how extreme an acceleration of progress can be.
“It seems highly unlikely that generation times can be brought arbitrarily close to zero. This provides an important kind of barrier to singular growth,” Ord writes. He suggests that training times for the next generation of models are likely to bottom out at some irreducible finite limit, which would prematurely end any period of singular growth. This damping could arise either because the system runs out of room for improvement as it nears some form of optimal performance per unit resource, or because the system strains under increasing size and complexity.
Possible asymptotic limits to AI growth
Ord lists several concrete kinds of limits that could cause MI progress to asymptote:
- Limits of intelligence itself: “Even an optimal reasoner would be neither omniscient nor omnipotent.” In other words, an ideal reasoner would still face intrinsic limits.
- Limits of intelligence per unit resource: even with optimal algorithms and hardware, available resources are finite and expanding beyond a local system is slow; Ord notes our solar system is only one star among roughly 200 billion in the Milky Way, and spatial expansion scales unfavourably (cubic slowdown).
- Limits of hardware: an optimal silicon chip might still be far below the physical limits of compute per unit resource.
- Limits of the algorithm: an optimal neural network could be substantially less capable than the best possible intelligence achievable with the same compute budget.
- Limits of training data: the data we have (and could acquire during RSI) lack much information in many domains, especially non‑verbalizable and highly contextual knowledge.
RSI weak links and timescales
Ord emphasises that, just as a task is composed of many sequential steps, RSI also depends on improvements across multiple sub‑components and abstraction layers. Each of these components likely has some hard limit, and both the speed at which a system approaches those limits and the absolute limits themselves will shape the appearance of any intelligence explosion.
He gives examples of feedback loops with very different characteristic timescales that could contribute to RSI: decades (for instance, designing a successor to EUV lithography), months (designing better pretraining), or seconds (improving scaffolding mechanisms).
Four phases of an intelligence explosion
Ord divides the potential growth trajectory into four phases:
- Initial exponential phase: doubling times are determined by human‑only research.
- RSI‑driven acceleration: RSI reduces generation times toward machine speeds, producing super‑exponential growth that outpaces human progress.
- Automated RSI at faster exponential: fully automated RSI continues at a higher exponential rate for a while but begins to saturate.
- Inflection and plateau: as the system nears its inflection point, the trajectory departs from exponential growth and follows a logistic curve toward a horizontal plateau.
Why this matters — implications for solar system expansion
Analyses like Ord’s help us think about the absolute rate at which a digital intelligence could increase its capabilities, and they implicitly bear on questions about energy harnessing and expansion into surrounding space (from near‑Earth resources to the inner and outer solar system). Even if singular growth is harder to achieve than sometimes assumed, Ord cautions this does not imply RSI is safe or that AI research will progress at a manageable speed.
“If the human‑only trajectory were A(t) and RSI sped this up to A(10t), we’d be getting a decade of human‑only progress each year, introducing many of the dangers — even without any change in the fundamental shape of the curve,” he writes.
Reference
The full analysis is available in Ord’s paper “The Dynamics of Intelligence Explosions” (arXiv).
