Customization lets developers adapt a general language model to specific tasks, domains or languages, but it normally requires infrastructure, tooling, GPU resources and domain decisions (which algorithm, which environment, how to evaluate learning). Open models such as the NVIDIA Nemotron 3 family (Nano, Super, Ultra) together with hosted platforms like Prime Intellect Lab make the full customization loop more accessible.
This article follows a practical tutorial that uses Prime Intellect Lab to customize NVIDIA Nemotron 3 Nano via reinforcement learning with verifiable rewards (RLVR) and produce a downloadable LoRA adapter. The local setup takes roughly five minutes.
What you build and why it matters
The workflow is simple: baseline evaluation, training, and reevaluation. The tutorial uses a Python Math environment as a small, verifiable example: the model may run Python tools (numpy, sympy, scipy) to perform calculations and must place the final answer inside a boxed response. The goal is to demonstrate how quickly and cheaply a short RLVR run can improve task performance.
RLVR and the Python Math environment
In RLVR the environment automatically verifies correctness and returns binary rewards (1 = correct, 0 = incorrect or missing). The Math environment limits the number of assistant turns (the example caps each rollout at five assistant turns; a direct response counts as one turn, an API/tool call plus final response counts as two). This discourages repeated tool calls without producing a final boxed answer.
Prerequisites
You need:
- a development environment with curl and a supported Python 3 installation;
- a Prime Intellect account with Hosted Training access and billing configured;
- internet access for the Prime CLI, environment packages, hosted models and adapter deployment;
- about 5–15 minutes for local setup.
Prime Intellect handles hosted rollouts, training and inference infrastructure, so you don’t need a local GPU cluster. Model availability and pricing can change, so treat the live CLI catalog as the source of truth.
Three steps to customize Nemotron 3 Nano
1) Get a baseline
Install and authenticate the Prime CLI, create a Prime Intellect Lab workspace, pin the Math Python environment version, and confirm the Nemotron 3 Nano model is available (example identifier: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16).
The tutorial runs a baseline evaluation with 32 held-out examples, one rollout per example, and a 5-turn cap. The initial results show poor performance: the baseline accuracy in the example is 21.9% (7/32), with a low average reward.
2) Train — create an RL LoRA adapter
Create a TOML config specifying the model, loss type (rl), max_steps (in the example 100), batch_size (32), rollouts_per_example (8), learning_rate (2e-5), lora_alpha (16), and the pinned environment (primeintellect/math-python@0.1.10 with dataset_name = "math" and max_turns = 5).
Launch the run with the Prime CLI. After confirmation the CLI returns a run ID and dashboard URL. The dashboard and CLI show reward curves, distributions and individual rollouts. Inspect a variety of rollouts (high, middle, low reward) to ensure the reward matches genuine correctness and that token usage or truncation do not bias the signal.
In the presented example a short 100-step run produced a visible reward improvement.
3) Evaluate learned behavior and deploy the adapter
When the run completes, Prime Intellect Lab uploads the final LoRA adapter. Confirm the adapter artifact is READY, then create a deployment for inference. Once the adapter reaches DEPLOYED, the adapted model identifier combines the base Nano model and the adapter ID.
Re-run the same evaluation (same 32 examples and the same sampling and environment settings) using the adapted model. The example shows a significant improvement: accuracy rose from 21.9% (7/32) to 90.6% (29/32), an absolute increase of 68.75 percentage points. Of the 24 prompts that changed outcome between baseline and final, 23 improved and one regressed. The total spend reported in the example was less than $5.
Cleanup is possible by deleting the deployment when it is no longer needed, avoiding further charges.
Training larger Nemotron models
The same workflow applies to Nemotron 3 Super and Nemotron 3 Ultra: verify availability in the live train models catalog, then replace the model identifier in the config with the chosen Super or Ultra model line and repeat baseline → train → reevaluate. The workflow and experimental discipline remain the same.
Why openness and reproducibility matter
Open weights are important but not sufficient: data, evaluation code and training recipes matter for understanding, reproducing and safely deploying models. NVIDIA publishes Nemotron 3 weights, training recipes and related developer resources, which help teams decide how their own customizations should differ.
Conclusion
Using Prime Intellect Lab, a short RLVR-based fine-tuning loop can produce a LoRA adapter for NVIDIA Nemotron 3 Nano that substantially improves performance on a verifiable Python Math task — in the example, lifting accuracy from 21.9% to 90.6% on 32 held-out problems with under $5 spent. The same, inspectable loop can be applied to larger Nemotron 3 models, enabling repeatable and auditable customization for real tasks.
For more details and resources, consult the Nemotron developer documentation, Prime Intellect Lab guides and the community channels referenced by the Nemotron project.
