NVIDIA and the Nemotron team have produced an NVFP4-quantized checkpoint for Nemotron 3 Ultra (550B) using NVIDIA Model Optimizer. The mixed-precision checkpoint reduces the model size from 1,121 GB in BF16 to 352.3 GB (a ~3.2× reduction) and yields up to 5.9× higher inference throughput on decode-heavy workloads versus the GLM-5.1 754B FP4 model, while matching BF16 accuracy across most benchmarks.
What NVFP4 is and why it matters
NVFP4 is a 4-bit floating-point quantization format introduced with the NVIDIA Blackwell architecture. As context windows grow and model weights become larger, efficient movement and storage of weights is critical for performance; quantization shrinks memory and bandwidth needs, increasing inference throughput and reducing hardware footprint.
Structure of the Nemotron 3 Ultra NVFP4 checkpoint
Not every layer in an NVFP4 checkpoint is stored in NVFP4. Layers are quantized to different formats depending on sensitivity and accuracy impact. Examples from Nemotron 3 Ultra (baseline BF16 → quantized precision):
- Embedding, Output classification layer, MTP layers: remain BF16
- MoE routed experts: BF16 → NVFP4
- MoE shared experts: BF16 → FP8 per-tensor
- Mamba mixer linears: BF16 → FP8 per-tensor
- Attention linears, Latent MoE, Mamba conv1d: remain BF16
- KV cache: BF16 → FP8
- Mamba SSM cache: FP32 → FP16 with stochastic rounding
This mixed precision reduced the model size from 1,121 GB to 352.3 GB, a ~3.2× shrink, cutting the hardware footprint substantially.
One checkpoint for multiple architectures
A key advantage of the Nemotron 3 Ultra NVFP4 checkpoint is that a single checkpoint can run on both NVIDIA Hopper and Blackwell hardware by converting weight formats at runtime. On Hopper (no native FP4 tensor cores) the serving framework switches to W4A16; on Blackwell it uses native W4A4. W4A16 on Hopper also preserves room for Multi-Token Prediction (MTP), while W8A8 would leave too little headroom.
Quantization challenges and methods
FP4 quantization has only eight positive representable values: 0, 0.5, 1, 1.5, 2, 3, 4, and 6. Choosing the block scale determines how original values map onto that grid. Bad scaling either flushes small values to zero or clips large values, harming accuracy.
- Max scaling (absmax): sets the scale by the block’s maximum absolute value. Simple but sensitive to outliers that compress other values.
- MSE scaling: searches for the scale minimizing average reconstruction error across a block. It reduced per-tensor weight error in experiments, but lower MSE did not always produce consistent downstream benchmark improvements.
In Nemotron 3 Super experiments, the best recipe combined MSE-based block scaling for weights with a per-tensor FP8 sweep and dynamic max-based scaling for activations.
Four-over-six scaling
NVFP4’s grid jumps from 4 to 6, so values in that gap are rounded aggressively. Four-over-six lets each block choose whether to scale to a maximum M=4 or M=6, picking the option that minimizes reconstruction error. Applied to Nemotron 3 Ultra’s FP4 routed-expert weights, four-over-six increased the global per-tensor weight scale by 1.75× and, across all 49,152 projection weights in the model’s 48 MoE expert layers, reduced the median reconstruction MSE by 16.4% versus standard max calibration. In the balanced 5.03 BPE setting it delivered 98.5% median recovery relative to BF16 (vs. max 96.8% and MSE 98.4%).
Bits-per-element and finding the operating point
Effective bits-per-element (BPE) is the average bits needed to store model weights: BF16 is 16 BPE, a mixed half-FP8 half-BF16 model is 12 BPE, and NVFP4 has overheads that push its minimum to about 4.5 BPE. Because layers vary in robustness to quantization, the configuration search is combinatorial.
NVIDIA Model Optimizer’s AutoQuantize (mtq.auto_quantize) accepts a target bit budget (e.g., auto_quantize_bits=4.8) and candidate formats, scores each layer’s sensitivity, and searches for a per-layer format assignment that meets the budget with optimal accuracy. For Nemotron 3 Ultra, a sweep across 4.85–7.19 BPE showed 5.03 BPE as the sweet spot judged by AA-LCR improvements.
How Nemotron 3 Ultra was quantized with Model Optimizer
Because Nemotron 3 Ultra (550B) is large, the team used parallelized quantization. Two supported paths exist: a Hugging Face Transformers flow and a Megatron-LM flow. The parallel Megatron-LM route converts the pretrained checkpoint to Megatron-LM format and invokes quantize.sh with an NVFP4 config; Megatron-LM shards the model across GPUs with EP = DP = 16 on 16×B300s so the calibration forward pass runs distributed.
Time comparison example:
- Hugging Face Transformers route: total ~120 min (load 40 min, calibration 85 min, export 42 min)
- Megatron-LM parallel route: total ~45 min (load <2 min, calibration 9 min, export 33 min)
Calibration uses the nemotron-post-training-dataset-v2 to fit per-block scales; the precision policy is config-driven and can be a built-in name or a YAML recipe that mtq.quantize() consumes.
Configs, recipes and tooling
NVIDIA Model Optimizer provides built-in NVFP4 configs such as NVFP4_DEFAULT_CFG and more selective presets (NVFP4_MLP_ONLY_CFG, NVFP4_EXPERTS_ONLY_CFG, NVFP4_OMLP_ONLY_CFG) to limit FP4 to particular modules. Under the hood, a recipe is an ordered list of rules matched to module-name patterns: weight quantizers are targeted via *weight_quantizer rules and activation quantizers via *input_quantizer rules. You can quantize weights only or both weights and activations, and disable quantization for specific modules.
The provided nvfp4-4o6.yaml is an example recipe that applies NVFP4 with four-over-six to routed-expert weights, keeps shared experts and Mamba projections in FP8, uses FP8 for KV cache, and leaves other parts in BF16. Four-over-six support is slated to land in NVIDIA Model Optimizer 0.46 (July release).
Reproducibility and launcher
The whole pipeline can be reproduced with Model Optimizer pointing at a Hugging Face model card or local path, selecting a built-in preset or custom recipe, then running quantize and export. The Model Optimizer launcher can automate the full PTQ and export workflow and has examples (Megatron-LM PTQ) that launch a Slurm workflow validated on four nodes with four NVIDIA Blackwell GPUs each.
Acknowledgments and resources
The work involved collaboration between the NVIDIA Model Optimizer and Nemotron teams, with contributions from Megatron-LM. Special acknowledgment goes to Asma Kuriparambil Thekkumpate, Jenny Chen, and Jinhang Choi for leading the NVFP4 implementation on Nemotron 3 Ultra. Documentation, recipes and examples are available in the NVIDIA Model Optimizer GitHub repository, along with Model Optimizer LLM PTQ documentation and the Nemotron 3 Ultra technical report and checkpoint materials.
