Research

AI-generated text

NVIDIA outlines post-training approach for scientific Nemotron models

NVIDIA has described a method for adapting Nemotron 3.5 Super VL to scientific tasks using experimental data and reinforcement learning.

NVIDIA outlines post-training approach for scientific Nemotron models

NVIDIA has outlined a post-training method for adapting Nemotron 3.5 Super VL to specialized scientific tasks. In its example, the model predicts the metabolite produced from a starting compound and a transforming enzyme, and its answers are evaluated against experimental data.

According to the company, the method addresses a significant challenge for autonomous AI agents conducting scientific workflows: unchecked errors can accumulate across dependent steps, while latency and computing costs also increase. Such agents can plan tasks, run simulations and assess results based on hypotheses expressed in natural language.

Nemotron 3.5 Super VL is part of NVIDIA Nemotron for Science, a collection of models, datasets, tools and recipes. Alongside the open, customizable multimodal model, it includes NVIDIA Nemotron Litmus-Bench V2.0, an open-source chemistry-reasoning dataset and benchmark designed to train and evaluate models on molecular properties. NVIDIA says the model achieves frontier-level accuracy on scientific benchmarks out of the box and solves scientific coding tasks quickly.

Turning experimental data into a training environment

The drug-metabolism example uses experimentally confirmed biotransformation reactions from MetxBioDB. Inputs and outcomes from a source CSV file are converted into question-and-answer pairs. Molecular structures are expressed using the text notation Simplified Molecular Input Line Entry System, or SMILES.

An automated verifier compares the model's answer with the experimental reference. This creates a reinforcement learning environment with verifiable rewards that needs no human preference labeling: the reward comes directly from the data. NVIDIA NeMo Gym supplies the environment infrastructure, while NVIDIA NeMo RL provides the post-training framework.

In the example, the verifier compares molecular fingerprints representing structural features of the predicted and reference metabolites using Tanimoto similarity. For the binary fingerprints used here, scores range from 0 to 1, with higher values indicating greater similarity. For other tasks, the scoring layer could use simple sequence similarity, cheminformatics libraries, molecular simulators or even an autonomous laboratory.

Reinforcement learning and model distillation

The direct training approach uses group relative policy optimization, or GRPO, with low-rank adapters, known as LoRA. The model reasons through the task step by step and submits its predicted metabolite through a tool call. The verifier scores the answer automatically. NVIDIA says the pipeline can be adapted to other domains where a numerical reward can be assigned.

According to the company, with a 16,384-token output budget, supervised fine-tuning improved Nemotron for Science accuracy by 12% in the drug-metabolism environment, and subsequent reinforcement learning produced a further 17% improvement. The reported evaluation examined mean molecular similarity.

When the starting model needs additional help or reinforcement learning converges slowly, distillation can be used first. A stronger teacher model generates solutions to the same tasks, and reasoning traces associated with answers that match or are sufficiently similar to the reference are used to fine-tune the smaller model.

In the example, Anthropic's Claude Sonnet 5.5 generated the training examples. From 1,934 training prompts, 1,123 high-quality reasoning traces were selected using a Tanimoto threshold of at least 0.7. One illustrated task required identifying N-demethylation by the CYP1A2 enzyme; the submitted metabolite fingerprint exactly matched the reference. The similarity score itself, however, measures the final molecular fingerprint rather than independently validating every step of the reasoning.

Putting the workflow into practice

NVIDIA's recipe begins by converting CSV records into training and validation examples in JSONL format. A domain-specific server stores the reference SMILES for each session and scores the first submission using Tanimoto similarity between 2,048-bit Morgan fingerprints with a radius of two. The environment and datasets are then connected to the NeMo Gym and trainer configurations.

The illustrated multi-node runs use 16 GPUs across four nodes: the GRPO example runs for up to 100 training steps, while the supervised fine-tuning example uses 55. The distillation pipeline scores the teacher model's responses and builds a supervised training dataset from examples that meet the chosen threshold.

The central design choice is the reward function, which must measure the desired scientific outcome. NVIDIA recommends starting with a narrowly defined task, validating the scoring method and using held-out evaluation data to determine whether post-training actually improves on the starting model.