Nvidia has published Nemotron 3 Ultra, a large language model designed for long-context, agentic tasks. It is the company's largest open-weights model so far: 550 billion parameters in total, with 55 billion active per token.
Capabilities and availability
- Input/output: the model accepts up to 1 million tokens of input; measured output speed is roughly 183 tokens per second.
- Availability: weights, training data and code are freely available under the OpenMDW-1.1 license. Chat access is offered via Perplexity Pro, and API access is available from Nvidia and other vendors; median pricing is about $0.60/$2.60 per million input/output tokens.
Architecture and training
Nemotron 3 Ultra scales up the design of the smaller Nemotron 3 Super and uses a hybrid mamba–transformer mixture-of-experts architecture. Key technical choices include:
- Mamba layers to handle long sequences with lower memory and compute cost than full self-attention, combined with a smaller set of attention layers for precise recall across long contexts.
- LatentMoE implementation: tokens are compressed into smaller representations and routed to a subset of 10 experts; multi-token prediction layers can generate more than one token at a time.
- Pretraining: two-phase pretraining on 20 trillion tokens — 15 trillion on broad knowledge and 5 trillion on higher-quality data, including 173 billion GitHub code tokens and synthetic legal and factual datasets. Training used a quantized 4-bit format, NVFP4, to reduce memory use and improve token processing efficiency.
- Fine-tuning: supervised fine-tuning was followed by reinforcement learning across reasoning, coding, agentic, chat, safety and usability tasks. The team trained more than 10 domain-specialized models that served as teachers in Multi-Teacher On-Policy Distillation; each teacher graded the student within its specialty and rewarded the student after every token. Distillation ran in two iterative rounds, rebuilding teachers from the improved student at the start of each round.
Features
- Three reasoning modes: off, regular and medium; a reasoning budget and support for tool use.
- Fine-tuned for open agent harnesses such as Hermes Agent and OpenClaw.
- Multilingual support for 12 languages.
Performance and independent benchmarks
- Artificial Analysis ranked Nemotron 3 Ultra as the highest-scoring U.S. open-weights model on its Intelligence Index, though it did not surpass some leading international open models.
- Intelligence Index scores: Nemotron 3 Ultra scored 47.7 using the recommended reduced-precision NVFP4 weights and 48.2 at full precision. For comparison, Google Gemma 4 31B (set to reasoning) scored 39.2 and OpenAI gpt-oss-120b (set to high reasoning) scored 33.3. Moonshot Kimi K2.6 led this set with 53.9.
- IFBench (instruction-following): Nemotron 3 Ultra scored 81.4%, placing third behind Grok 4.3 (83.3%) and Grok 4.20 0309 / MiniMax-M3 (both 82.9%).
- Long-context recall (Nvidia Ruler, 1 million token context): 95%. Agent productivity (PinchBench): 91%, matching Kimi K2.6. Agentic coding (Terminal-Bench 2.0): 54%, behind Moonshot Kimi K2.6 (67%) and Z.ai GLM 5.1 (64%).
- Speed: in tests by Artificial Analysis running on multiple providers, Nemotron 3 Ultra was about three times faster on average (around 183 tokens/sec) than comparable open-weights models such as Moonshot Kimi K2.6 and DeepSeek V4 Pro.
Why this matters
Recent leading open-weights models for building agents have often come from Chinese developers (for example Kimi K2.6, Qwen3.5, DeepSeek V4, GLM-5.2). Nemotron 3 Ultra returns a high-capability, fast and fully documented U.S. open model to the field, providing a basis for developers to build agentic systems.
Related Nvidia releases
Shortly before Nemotron 3 Ultra, Nvidia released other products aimed at agentic performance: the Vera CPU (its first processor designed for agentic work), RTX Spark (a Windows PC chip for on-device agents) and Cosmos 3, an open world model for generating training data for robots, self-driving cars and other agents.
Summary
Nemotron 3 Ultra is Nvidia's largest and most documented open-weights model, oriented toward long-context and agentic tasks. Its combination of speed and open availability gives developers a practical base for agent development, even though it does not top every performance leaderboard.
