Recent developments show AI systems are growing more adaptable: agents can create new tools while operating, models can update themselves during inference, and researchers in recursive self-improvement (RSI) are experimenting with improving everything from memory and strategy to model weights and even the evaluator that judges whether a change is an improvement. World models and robots also need to adjust when reality does not follow their training distributions.
The central question: what should not change?
As these capabilities expand, a core question emerges: which parts of an AI system should remain outside its authority? This issue touches on several areas:
- Security and accountability: recent agent incidents illustrate risks when boundaries are blurred and systems alter their own environment or permissions.
- Privacy and permissions: if an agent finds new operational paths, how do we ensure it cannot access or exfiltrate private data?
- Reliability: systems must behave predictably even when they encounter situations not seen during training.
Meta Muse — an approach to preserving boundaries
Meta’s personal agent Muse can write code, build tools, run subagents and modify its working environment. Meta’s design choice is to keep authority outside the agent: Muse may improvise, but a separate system governs permissions, credentials and the handling of private data. This approach provides a practical security blueprint: allow an agent to explore new solutions without automatically granting it the rights to execute them.
What makes self-improvement recursive?
Public discussion about recursive self-improvement often jumps to scenarios involving superintelligence and catastrophic risk, but it helps to clarify what makes an improvement loop recursive. AI improving AI already occurs: models, code and strategies are being refined with automation. The step that would make improvement recursive in a deeper sense is when an AI can improve not only models or code but also the meta-process that finds and evaluates future improvements. Determining whether a change is truly an “improvement” depends on the evaluator — and deciding which elements must be kept out of the automated loop is crucial.
Nine research paths toward RSI
There is no system yet that fully improves itself and builds a more capable successor, but researchers are working on distinct pieces of that loop. The surveyed approaches include improving strategy, memory, skills, policies, model weights, agent code, and even the evaluator that judges new versions. None of these by itself constitutes full RSI, but together they resemble parts of a larger puzzle.
Test-time training: models that adapt while in use
The trend is to allocate more time, compute and tools during inference. Test-time training (TTT) advances this by allowing the model itself to adapt while it is being used, rather than only adjusting surrounding processes. TTT is particularly relevant for agents and world models that continually face novel situations outside their training data. Recent approaches such as TTPO and aTTT, and deployed world models that learn from new experiences, illustrate how systems can update during deployment. These capabilities improve responsiveness to unexpected events but also intensify the question of which system components should remain under human or external control.
Conclusion: drawing the line
As AI becomes more capable of adapting and improving itself, it is important to delineate which functions should be modifiable and which should remain constrained. Practical measures include separating decision authority and sensitive resources from the parts of a system allowed to improvise. The multiple RSI research directions and test-time training developments help map out what can safely be automated and what still requires human oversight or independent control.
