NVIDIA has released Nemotron 3.5 Lightning, an open 30B mixture‑of‑experts (MoE) model that typically runs with 3B active parameters at inference. Lightning is designed for the execution layer of always‑on AI agents where high call volumes, tool invocations, result validation and subagent delegation dominate compute and latency budgets.
Why Lightning is aimed at agent execution
MoE architectures route each token to only a subset of the model’s experts, so per‑token computation involves a fraction of the total parameters. This provides the effective capacity of a larger dense model at a smaller compute cost, making Nemotron 3.5 Lightning suitable for high‑volume, low‑latency agent workloads. Lightning is the smallest member of the Nemotron 3 family and inherits several techniques validated across that family.
Key technical features
- Model size: 30B MoE (30 billion parameters) with about 3B active parameters during typical runs.
- Speculative decoding: multi‑token prediction (MTP) was included in the model’s pretraining and later boosted in a dedicated phase. Two draft models ship with Lightning: DSpark (recommended for DGX Spark inference and low‑concurrency data center workloads) and DFlash.
- Quantization: the release includes NVFP4 and BF16 checkpoints, using the NVFP4 kernels that power Nemotron 3 Ultra on NVIDIA Blackwell, Hopper and Ampere GPUs.
- Harness‑optimized training: trained for common agent harnesses like OpenClaw and Hermes Agent, and integrated with the NemoClaw open source security and management stack.
Performance and efficiency
NVIDIA reports that Nemotron 3.5 Lightning sits on the accuracy‑vs‑speed Pareto frontier for small open models on the Artificial Analysis Intelligence Index, which aggregates nine evaluations across agentic tasks, coding, scientific reasoning and general intelligence. NVIDIA highlights up to 4× output speed compared with similar‑sized models. On PinchBench, Lightning reached 86% accuracy while completing 10,000 tasks about 30% faster than Qwen3.6 35B at comparable accuracy. Higher inference throughput and token efficiency help always‑on agents complete routine workloads faster and spend fewer tokens per unit of useful work.
How Lightning maintains speed without losing accuracy
Lightning combines several approaches:
- Speculative decoding and built‑in MTP: MTP was baked in during pretraining and improved with a dedicated MTP‑boosting phase.
- Draft models: DSpark and DFlash are provided to match different concurrency and workload profiles; MTP draft length interacts with concurrency to determine optimal settings.
- NVFP4 quantization: specialized kernels enable efficient deployment across data centers and local DGX Spark systems.
Customization and fine‑tuning
The release purposefully makes Lightning easy to adapt. Smaller models fine‑tune faster and cheaper on modest hardware. Weights, training data and recipes are released under the permissive OpenMDW‑1.1 license. Supported customization workflows include:
- LoRA or full SFT via NeMo Automodel and NeMo Megatron Bridge.
- Reinforcement learning and environment‑based evaluations with NeMo RL and NeMo Gym. The release also includes Nemotron‑RL Agentic Terminal Pivot, an open reinforcement learning dataset used for training some coding agent capabilities.
Routing with NeMo Switchyard
While frontier reasoning models (for orchestration and complex planning) get much attention, smaller models like Nemotron 3.5 Lightning handle the routine calls that dominate an agent’s token budget (git pull, tool output validation, formatting results, etc.). NVIDIA NeMo Switchyard provides model routing and orchestration so requests can be sent to the most capable and efficient model: plans can route up to frontier models, while execution requests route down to Lightning.
Local and cloud deployment, ecosystem
Nemotron 3.5 Lightning is intended to run on local systems (NVIDIA Jetson, GeForce RTX 5090, DGX Spark) as well as in data centers. It is compatible with tooling such as LM Studio, llama.cpp, Ollama and Unsloth, and NVIDIA provides deployment guides for vLLM, SGLang and TensorRT‑LLM. The model is supported by a broad partner ecosystem spanning post‑training services, inference software, harnesses and cloud platforms.
Where to get it and how to start
The model is fully open: weights and related artifacts are available on Hugging Face and ModelScope, and you can try it on build.nvidia.com or through OpenRouter. NVIDIA also publishes a Nemotron 3.5 Lightning cookbook, deployment guides and Switchyard documentation to help developers adopt the model.
Conclusion
Nemotron 3.5 Lightning targets the execution heavy layer of always‑on agents by combining MoE routing, speculative decoding and NVFP4 quantization to increase throughput and token efficiency without sacrificing accuracy. Together with NeMo Switchyard and the NemoClaw agent stack, the model is positioned to be the routine‑workhorse in multi‑model agent systems where frontier reasoners and smaller execution models cooperate for efficient use of compute and tokens.
