A research team proposes a sleep-inspired framework for continual learning in large language models (LLMs). The "sleep" paradigm alternates between an active interaction phase and an offline consolidation phase in which fragile short-term context is distilled into stable, long-term parameters. The approach aims to enable continuous acquisition of new knowledge and capabilities while avoiding catastrophic forgetting.
Motivation
Current LLMs are typically pretrained and then deployed with a fixed knowledge cutoff and limited capacity to retain new information beyond the immediate context window. Continual parameter updates can cause catastrophic forgetting, while relying solely on in-context learning (ICL) means newly acquired information vanishes when the session ends. The sleep framework addresses both limitations by providing an automated offline consolidation mechanism that integrates new facts into parametric memory without degrading prior abilities.
The sleep paradigm: two phases
The model lifecycle is split into two primary states:
- Active (wake) phase: the model processes inputs and stores temporary information in fast, high-frequency memory components during live interactions.
- Sleep phase: the model stops consuming new inputs and redirects computation to memory consolidation and self-improvement, activating pre-allocated Mixture-of-Experts (MoE) memory blocks and updating numerical weights using self-generated practice examples.
No structural change to the model core is required; the framework uses predesignated MoE blocks and training rules to perform offline consolidation.
Memory consolidation mechanisms
Memory consolidation in the sleep framework relies on two main technical mechanisms:
- Parameter expansion and a continuous memory hierarchy
- To avoid catastrophic forgetting, the model dynamically increases capacity during sleep by unlocking new, low-rank MoE experts that are incorporated as stable, low-frequency memory blocks. This lets the model write incoming knowledge without overwriting previously consolidated parameters.
- Knowledge seeding (upward distillation)
- Fragile short-term traces (for example, in-context memories) are transferred into long-term parameters via Generalized Knowledge Distillation (GKD) combined with reinforcement-learning (RL)-based imitation learning:
- On-policy distillation: the expanded student model learns token-level feedback from teacher distributions generated on self-sampled sequences, aligning consolidation with the trajectories the model will produce at inference time.
- Learning to imitate (LTI): the student is rewarded based on semantic and absolute similarity (e.g., Levenshtein distance) when completing prefixes of teacher-generated text.
- Synaptic pruning
- After consolidation into lower-frequency modules, the high-frequency temporary parameters are reset to free capacity for future active sessions.
Dreaming for self-improvement
During a REM-like dreaming stage, the framework runs an autonomous self-improvement loop:
- Random expert selection: the MoE router randomly activates non-standard experts during synthetic data generation, forcing domain mixing and revealing latent patterns.
- Curated rehearsal: the model generates synthetic "dream" samples and filters them using gradient-based data selection to prioritize samples likely to boost overall capability.
- RL refinement: the model performs supervised fine-tuning on top-rated dream samples and evaluates whether generation improves downstream task performance.
Theoretical foundations
The authors provide an analysis framing knowledge seeding as a structured, low-rank compression problem: rather than copying every detail of short-term traces, the model identifies a compact set of directions that best preserve on-policy behavior and semantic qualities emphasized by the learning objective. On-policy distillation reduces mismatch between training and actual inference trajectories, while a semantic reward emphasizes preserving meaning and task-relevant behavior rather than merely matching token probabilities. Parameter expansion supplies the capacity to store these selected memories without overwriting earlier consolidated knowledge.
For dreaming, random expert activation and reward-based sample selection broaden the set of gradient directions explored during synthetic generation. Gradient-based filtering emphasizes synthetic experiences that carry the most useful learning signal, improving conditioning of gradients and producing a more balanced optimization landscape. In this framing, dreaming is a deliberate process of diversifying, selecting, and consolidating updates rather than simple data augmentation.
Empirical results (key figures)
The sleep framework demonstrated improvements across several tasks in the reported experiments:
- Continual translation: on unseen languages such as Manchu and Kalamang, the sleep approach maintained a near-zero performance drop in multi-stage evaluations, whereas baselines like ICL, Cartridges, and supervised fine-tuning (SFT) exhibited severe catastrophic forgetting during sequential learning.
- Long-context understanding (BABILong benchmark): the sleep method achieved near-perfect scaling up to 10 million (10M) tokens and outperformed strong baselines, including GPT-4 and Titans, by consolidating long-context information into parametric memory during sleep.
- Factual knowledge incorporation (SQuAD and continued pre-training, CPT): the sleep method achieved 48.9% on single-task evaluation and 46.2% on CPT, outperforming baseline methods including the base model, SFT, and Self-Adapting Language Models (SEAL) while integrating new factual information without degrading existing representations.
- Mathematical reasoning (AIME 24/25 and HMMT 25): the sleep-enhanced base model reached accuracy scores of 79.2%, 69.0%, and 46.1% on the listed benchmarks, surpassing baselines such as SFT, group relative policy optimization (GRPO), and on-policy self-distillation (OPSD). These results suggest that combining memory consolidation with imitation learning benefits complex reasoning.
Significance and conclusions
The sleep paradigm represents a pathway toward autonomous, continually learning AI systems. By alternating active learning with offline rest, LLMs can retain short-term memories, abstract new skills into long-term parametric structures, and self-improve without catastrophic forgetting. The reported empirical and theoretical results indicate that memory consolidation plus dreaming-style rehearsal can expand a model's ability to adapt and preserve knowledge over very long contexts.
Authors and acknowledgements
The research was conducted by Ali Behrouz (Google and Cornell University), Farnoosh Hashemi (Cornell University), Adel Javanmard (Google and University of Southern California), and Vahab Mirrokni. The authors thank Yuan Deng and Corinna Cortes for reviewing the work and providing suggestions, and Mark Simborg, Kimberly Schwede, and Tim Herrmann for assistance in preparing the accompanying blog post.
