Researchers at the Massachusetts Institute of Technology (MIT) and Columbia University developed a game‑theoretic model to study whether two competing firms can coordinate to slow development of powerful AI systems. Their paper, titled Racing to Ruin, asks “why exactly is coordination hard? And what would it take?” and concludes that two central ingredients for a stable coordinated slowdown are some degree of transparency about technological progress and the ability to model rivals as trustworthy, rational actors.
What the paper models
The authors construct a simple R&D competition between two dominant firms (a duopoly) under the shadow of disaster. As frontier firms scale up the technology, they increase the hazard of an event that permanently reduces all firms’ flow payoffs to zero. That hazard is a known function of the firms’ technology levels and arises from developing the technology, not from using it.
Main analytical findings
When monitoring is sufficiently precise, every equilibrium ends in finite time. However, a new temptation emerges: each firm prefers to stop second and will only exit after confirming that the rival has stopped. To stop first — without knowing if the rival has stopped — is a gamble on both the rival’s type and on rapid arrival of news: if the rival is rational, it will stop upon receiving the news of the first firm’s stop, and otherwise it will not.
Trust and transparency interact differently depending on the coordination structure. In sequential coordination, a firm must stop first, gambling that a rational rival will reciprocate once the news arrives; faster information increases the reward of such reciprocation. In simultaneous coordination, firms must be insufficiently tempted to keep racing and should stop only after seeing the rival actually stop — faster news makes both stopping first and waiting for verification more attractive.
Transparency has a two‑edged effect: faster detection lowers the cost of waiting for confirmation that a rival has stopped rather than stopping unconditionally. Thus, at intermediate trust, increasing transparency can initially destroy the early‑stopping equilibrium (by making the free‑riding deviation attractive) before restoring it once detection becomes fast enough to make stopping self‑enforcing.
The key conclusion — avoiding races to ruin
The paper identifies three regimes by trust level. With low trust, every equilibrium races to ruin: the disaster arrives with probability one. With intermediate trust, both immediate stopping and racing to ruin can be equilibria. With high trust, in every equilibrium the probability that two rational firms race forever vanishes quadratically in the prior odds ratio of rationality.
Why this matters
If policymakers and industry stakeholders want any realistic prospect of pausing or slowing the development of powerful AI systems, the analysis implies they will need regimes for transparent information sharing by companies about the state of their AI development and credible verification tools to ensure the shared information and any slowdown actions are genuine. The authors note parallels to historical arms‑control approaches for nuclear weapons, where trust is paired with verification.
Conclusion
Racing to Ruin shows that coordinated slowdowns are not impossible, but they depend sensitively on monitoring precision and mutual trust. Building institutional mechanisms that increase transparency and enable reliable verification will be critical to prevent potentially catastrophic competitive dynamics in AI development.
