SPADE (Self‑Play in Adaptive Synthetic Executable Environments) is a general framework in which synthetic environment generation and agentic capability co‑evolve through self‑play. The approach produces game‑like, executable training environments that a large language model (LLM) alternates between designing and solving. The idea is to use a powerful model to bootstrap the creation of diverse training data that can in turn be used to further train models.
Who built it?
SPADE was developed by a multi‑university team including researchers from the University of Washington, Stanford University, Northeastern University, Carnegie Mellon University, Massachusetts Institute of Technology, National University of Singapore, Seoul National University, Stevens Institute of Technology, and the University of Chicago.
How the framework works
In SPADE the LLM plays two roles:
- Environment Designer: writes complete, long‑horizon training environments as executable code (for example puzzles or simulated lab tasks).
- Reasoning Agent: learns to act within those generated environments and tries to solve the tasks.
The Environment Designer may attach privileged hints (h) to an environment — task‑relevant information such as a partial solution sketch or a key structural observation. Revealing h to the Reasoning Agent makes the environment easier; the difference in the Reasoning Agent’s return with versus without h defines the Environment Designer’s hint‑based regret reward. In other words, the reward signal for designing environments comes from how much the hint changes the agent’s performance.
Environments are represented as Python programs with a Gym‑style interface, which unifies single‑turn reasoning and multi‑turn agentic tasks and transforms environment design into a learnable RL‑trained component for continual, open‑ended self‑improvement.
Models, training and evaluation scale
The authors trained three Qwen3 backbones to test SPADE: Qwen3‑4B‑Instruct‑2507, Qwen3‑8B, and Qwen3‑30B‑A3B‑Instruct‑2507. As expected, the largest model, Qwen3‑30B, performed best. Each model was tuned with GRPO for 400 rollouts, with each rollout containing 25 environments.
Evaluation used multiple benchmarks, including AIME, GPQA, LCB, and environments inside Reasoning Gym. The system generates two main kinds of environments — game environments and tool‑use environments — and SPADE improved performance on both categories.
Quantitative results
For game environments on the 30B‑A3B backbone, SPADE reached a suite average of 58.3, which is +8.1 over the base model and +5.3 over the strongest fixed‑environment baseline. The same approach applied to tool‑use environment design yielded consistent improvements across all backbones.
Why this matters
SPADE is a practical form of advanced synthetic data generation: it enables researchers to use whatever high‑capability model they have to hand to produce a broader, richer set of training environments for subsequent training. Repeatedly swapping the underlying generator (for example across frontier models from different providers) could further increase environment diversity. That reduces the cost and effort of assembling large, varied datasets. At the same time, the authors note this does not create unlimited self‑improvement beyond the creative capacity of the base model used to generate environments.
Availability
The paper "SPADE: Self‑Play in Adaptive Synthetic Executable Environments" is available on arXiv, and the code and model checkpoints are published in the SPADE (spade‑rl) GitHub repository.
Brief summary
SPADE makes environment design an RL‑trainable, executable program generation process and demonstrates that self‑play between environment design and reasoning agents can yield measurable gains on both game and tool‑use tasks, particularly at the 30B model scale.
