Today, NVIDIA released the full NVIDIA Run:ai platform as open source software, making production-grade AI workload and GPU orchestration capabilities available to customers, developers, and ecosystem partners. This move extends the NVIDIA DSX AI Factory Platform by establishing Run:ai as an open foundation that the broader ecosystem can build on to bring differentiated solutions to market faster.
Over recent years, NVIDIA Run:ai has helped organizations run AI workloads on Kubernetes at scale by dynamically allocating GPU resources and managing workloads across the full lifecycle, from data processing and training to inference. Open-sourcing the platform enables DSX ecosystem partners and AI cloud providers to increase workload throughput, improve GPU utilization, and operate shared AI infrastructure more efficiently.
What was released
The open-source release includes the complete NVIDIA Run:ai platform plus four components that can be adopted independently:
- KAI Scheduler
- GPU Fractioning Operator
- KAI Resource Management
- Workload Map
Together with additional platform code, these elements form the integrated NVIDIA Run:ai platform. Platform teams and partners can adopt the full stack, build on it for their own products and services, or integrate individual components according to their needs.
How to adopt Run:ai
The full source code is available in the NVIDIA Run:ai repository within the NVIDIA DSX AI Factory GitHub organization. Teams can inspect, contribute to, or extend the code, or use the modular components independently. The complete open-source platform is also published on NVIDIA NGC as a Community Product, with a Helm chart for installing the integrated stack.
Note: the Run:ai source code is governed by applicable open-source licenses. Downloading the Community Product from NGC requires acceptance of the NVIDIA Software License Agreement and product-specific terms for NVIDIA AI products. No paid NVIDIA Run:ai subscription or NVIDIA support entitlement is required for the Community Product; enterprise support is not included.
Ecosystem integrations and partner offerings
Open-sourcing Run:ai gives NVIDIA DSX partners direct access to the orchestration innovations for use in their own DSX AI Factory solutions. Examples of partner integrations and offerings include:
- Crusoe: integration of Run:ai with Crusoe Managed Kubernetes to combine GPU orchestration with managed Kubernetes and GPU infrastructure.
- Mirantis: extension of k0rdent AI with Run:ai to provide a Metal-to-Model AI factory stack that automates provisioning and maximizes GPU utilization.
- Nebius: offering Run:ai as a one-click application deployment in Nebius AI Cloud on top of Nebius Managed Service for Kubernetes.
- OpenNebula Systems: integration with the OpenNebula AI Cloud Platform via OneKS for provisioning and lifecycle management in open and sovereign clouds.
- Rafay: delivering Run:ai as a fully managed service, automating the entire stack from GPU infrastructure to control plane and tenant management.
- Saturn Cloud: integration to enable white-label AI inference services via Token Factory with per-token metering and automated billing.
- Spectro Cloud: integration into PaletteAI so customers can deploy and manage Run:ai with their existing lifecycle automation.
- vCluster: providing pre-validated deployment templates to create tenant-isolated environments while Run:ai schedules across a single GPU fleet.
For NVIDIA Enterprise AI Factory deployments, Run:ai remains available through NVIDIA-supported commercial offerings (including as a standalone product, via NVIDIA Mission Control and NVIDIA AI Enterprise, and selected cloud marketplaces). Availability, licensing, and support terms vary by offering.
Modular open-source components
The release details four main components and how they are intended to be used alongside the rest of the platform.
KAI Scheduler — the Run:ai scheduling engine
KAI Scheduler is a Kubernetes-native scheduler powering NVIDIA Run:ai. NVIDIA released KAI Scheduler under the Apache 2.0 license and contributed it to the Cloud Native Computing Foundation (CNCF) as a Sandbox project. KAI is the upstream engine behind Run:ai and scheduler development happens in the community repository.
KAI Scheduler is purpose-built for large-scale GPU workloads and provides:
- Gang scheduling with prescheduling in-memory simulation for all-or-nothing multipod placement
- Fair-share (including time-based fair-share) and quota allocation
- Topology awareness and bin-packing at scheduling decision time
- GPU sharing modalities (MIG, MPS, time sharing, HaMi, CUDA 13.4 MPS fractioning)
- Dynamic Resource Allocation (DRA) support
KAI can be used for diverse AI workloads that require fair access to cluster resources and topology-aware placement; it can be deployed alongside existing stacks without requiring the full Run:ai platform.
GPU Fractioning Operator — CUDA-native GPU fractions on Kubernetes
Run:ai has long supported fractional GPU allocation so multiple workloads can share a GPU with hard memory isolation. The GPU Fractioning Operator brings CUDA-native GPU fractioning to Kubernetes and replaces the prior proprietary implementation with the native capability available in CUDA 13.4.
The CUDA driver enforces hard GPU memory limits for each fraction; each workload sees only its allocated GPU memory. The operator provides the Kubernetes integration layer and a metrics exporter for per-pod visibility. It is scheduler-agnostic but integrates with KAI Scheduler to convert capacity into placement decisions.
Use the GPU Fractioning Operator for notebooks, development sessions, and smaller models where sharing underutilized GPU capacity is beneficial.
KAI Resource Management — the organizational layer
KAI Resource Management governs who gets what within an organization. It provides:
- Projects and departments mapping
- Resource management with organizational hierarchies for allocation, guarantees, sharing, fair-share and quotas
- Node pools for grouping and targeting heterogeneous hardware
This layer lets platform teams manage GPUs according to organizational needs (teams, departments, initiatives) and replace ad-hoc quota tracking with programmatic guarantees. It can be combined with KAI Scheduler.
Workload Map — a workload-management contract
Workload Map defines a consistent contract Run:ai uses to represent any workload type. KAI Scheduler consumes it for scheduling and gang semantics; other platform components use it for discovery, monitoring, and lifecycle management.
Because AI workloads on Kubernetes are often CRD-based (PyTorchJob, RayCluster, JobSet, KServe InferenceService, Dynamo deployments, or custom CRDs), Workload Map provides a single way to describe pods, components and roles, submit and configure workloads, schedule with gang and topology awareness, observe status and health, and apply policy at admission or runtime. It ships as a Go library that also works offline on unstructured objects.
Use Workload Map for platforms and tools that must understand and manage multiple Kubernetes workload types without hardcoding each CRD.
Related open-source projects
The announcement also highlights complementary open-source projects that address adjacent AI infrastructure problems:
-
Model Streamer: reduces cold-start latency by streaming model weights directly into GPU memory. Published benchmarks on eight NVIDIA H100 VMs show 3x–6x faster loading than the default loader and a steady ~80 Gbps compared with ~40 Gbps for the default path. Several storage vendors and cloud providers have published independent improvements (examples include WEKA, VAST Data, and DDN measurements).
-
Grove: a Kubernetes API for orchestrating multicomponent workloads (agentic pipelines, disaggregated serving, mixture-of-experts training) with topology-aware autoscaling and hierarchical gang scheduling. Grove is a modular part of NVIDIA Dynamo and supports large-scale training and inference pipelines.
Getting started
NVIDIA Run:ai is available now as open-source software in the NVIDIA DSX AI Factory GitHub organization. Key resources include the full platform, KAI Scheduler, KAI Resource Management, GPU Fractioning Operator, and Workload Map, plus related projects Model Streamer and Grove.
NVIDIA invites community contributions, issues, and feedback as it advances open development of the next generation of AI workload and GPU orchestration.
