Hugging Face now provides a dedicated spot on the Hub for reinforcement learning (RL) environments: an RL Environments filter and framework-specific tags that let dataset repositories be discovered and run as environments.
What is an RL environment on the Hub?
An RL environment gives tasks to an agent, responds to actions with observations, and scores outcomes. Rewards can be used to evaluate agent performance or as learning signals during training. On the Hub, an RL environment is simply a dataset repository tagged rl-environment that appears in the new filter.
Hugging Face separates environments into two broad parts: tasksets (the task data) and runtimes/verifiers (runtime and scoring code). This release focuses on tasksets: the Hub stores and versions task data while frameworks supply runtime and verifier implementations when needed.
What changes in practice?
- RL Environments filter: visit huggingface.co/datasets?other=rl-environment to see every dataset tagged rl-environment, regardless of which framework it works with.
- Framework tags: four environment frameworks are registered as dataset library tags — harbor (Harbor), verifiers (Verifiers), openenv (OpenEnv) and nemo-gym (NeMo Gym). Each tag adds the framework icon on a dataset page and generates a snippet in the "Use this dataset" panel.
- Multiple tags per repo are supported: tags describe compatibility and are not mutually exclusive. Each listed framework must support the files in the repository; adding a tag does not convert files.
Rationale
Many RL papers and frameworks use their own registries or custom loaders, which fragments published environments: an environment published for one framework may not be loadable by others. Hugging Face argues that environments are data (tasks, tests, containers, reward rules) with a runtime on top. Since the Hub already hosts, versions, gates, previews, and serves data to millions, it should be the canonical place for task data and metadata indicating how to run it — rather than creating a separate environment registry.
From dataset repository to agent run
The Hub hosts task data; frameworks provide runtime and verifier code. At runtime, an agent exchanges actions and observations with an environment and a verifier assigns rewards for evaluation or training. The broad flow:
- Dataset repository on the Hub: holds tasks/ and optionally runtime and verifier files.
- Framework loads the files and supplies missing runtime/verifier implementations.
- Execution happens locally or on a supported cloud backend, with the agent interacting with the environment and receiving a verifier score.
Hugging Face Jobs can run cloud workloads, and Hugging Face Sandboxes (built on Jobs) provide interactive command execution. Tags indicate compatibility and generate loading commands; adding a tag does not start a job or sandbox.
How to run environments with supported frameworks
The announcement includes example commands for several frameworks. Key examples:
-
Harbor: can load task directories from a Hub repository; the oracle agent runs the task's reference solution and the verifier scores the result. Example uses harbor==0.21.0 to run terminal-bench-2.1 and view results, including reward, verifier output and logs.
-
Verifiers: the Verifiers Harbor integration runs the same task directories in different runtimes (e.g., Docker) and supports different harnesses, including a minimal bash harness.
-
OpenEnv: OpenEnv's Harbor integration can run the same task directories with agents such as OpenCode and return the verifier's reward alongside the agent trace. Example: openenv[harbor]==0.7.0 runs a task in Docker and writes rollout.json; you can read reward, n_turns (model calls), and error fields from that file.
-
NeMo Gym: supports evaluation and RL training. For example, the Structured Outputs dataset pairs prompts with JSON schemas, and its verifier rewards schema adherence (not factual correctness).
Note: a reward value of None means no verifier reward was produced; check the error field before labeling a run as a model failure.
Examples already on the Hub
Hugging Face opened PRs to tag several existing environments; maintainers can merge those PRs so their environments show up in the filter. Examples listed include:
- Harbor: BeyondSWE (benchmark as Harbor task directories), Terminal-Lego (Terminal-Bench-style tasks), Harbor-Mix (100 hard agentic tasks), NatureBench (90 tasks prebuilt for Harbor).
- Verifiers: Reverse-Text-RL, Multi-SWE-RL-Verified (2,232 of 4,703 rows passing gold-patch validation across several languages), R2E-Gym-Subset-Verified, Scale-SWE-Verified (17,202 of 20,181 Python issue-resolving tasks giving a clean reward signal).
- NeMo Gym: Workplace Assistant (690 business tasks, 26 tools, five databases), Structured Outputs, CFBench (multilingual constraint following), SysBench (multi-turn system message following).
A single repo and its discussion tab can now be the place where issues with tasks are reported and fixed, so a fix in the repo benefits all compatible frameworks.
How to tag your environment
Add the rl-environment tag and any framework tags to your dataset card YAML header. Example:
pretty_name: Terminal-Bench 2.0 tags:
- rl-environment
- harbor
- verifiers
That is the integration: keep rl-environment and list each framework that can load your files. If your environment works with a framework not yet registered, open a PR to the list of supported libraries. The docs contain the full reference.
What comes next
The first release generates one default snippet per framework. Next steps include per-config snippets (so a repo with multiple task sets shows the correct command for each) and structural detection for frameworks with strict layouts. The team is also experimenting with custom task UIs.
The longer-term goal is automatic framework tagging everywhere; OpenEnv already does this on upload. Framework maintainers are encouraged to add tags in their push path so published environments become visible to everyone.
Recommendations
- If you train agents: browse the RL Environments filter on the Hub.
- If you build environments: publish and tag them (rl-environment plus compatible framework tags), include the files, a working run command, and the rule that produces the reward.
- If your framework is missing: contribute it to the list of supported libraries.
The change aims to let a single dataset repo serve users across major environment frameworks, reduce porting overhead, and centralize discovery and versioning on the Hub.
