Agent Substrate is an open-source, security-hardened, zero-trust runtime for sandboxed AI agents that has been donated to the Cloud Native Computing Foundation (CNCF) Sandbox. It aims to run millions of agents and tools at up to 10× higher density than standard containers by multiplexing many short-lived, stateful sessions across a small pool of Kubernetes worker pods. Idle sessions are checkpointed (memory and filesystem) and later restored onto whichever worker becomes available.
Why this matters
Traditional Kubernetes workflows assume a modest number of long-lived service instances. Agent-style workloads behave differently: an agent performs a brief step (run a shell command, edit a file, call an LLM), then waits—sometimes milliseconds, sometimes minutes—before the next step. Spinning up a new Pod for every step incurs multi-second latency (API server, scheduler, kubelet, image pulls, etc.), or platforms must keep a warm Pod per session and pay for mostly idle compute. Both choices are inefficient, and the same constraints affect RL post-training and other agentic workloads.
How Agent Substrate works
Agent Substrate uses Kubernetes to host a pool of ready worker pods but runs a separate control plane that maps a large set of actors (agents and their environments) onto that smaller worker pool. When an actor becomes idle, the runtime snapshots it out of its sandbox (gVisor or a microVM) and frees the worker. On the next request, the runtime restores the actor—including its memory and filesystem—onto any free worker. That restore avoids a new round trip through the Kubernetes API server, scheduler, or kubelet. Worker pods remain ordinary Kubernetes Pods, so cluster scheduling, quotas, and placement policies still apply.
Key workloads driving adoption
- Coding assistants and agent serving: Agents alternate between model calls and local environment work (cloning repos, editing, building, running tests). Users expect persistent environments and immediate responses, which is costly if each session requires a dedicated warm sandbox.
- Reinforcement Learning (RL) post-training: GPU trainers operate alongside large fleets of CPU-heavy environments (sandboxes, compilers, simulators). Rollout bursts create bursty sandbox demand that can grow from hundreds in experiments to many hundreds of thousands in production; cold starts in environments can leave GPUs idle.
- GPU sandboxes for research: Researchers need short-lived, isolated GPU sandboxes to compile CUDA, benchmark kernels, or run numerical workloads; these require fast starts and fair sharing of limited GPUs.
Upstream progress and planned work
All work is public in the Agent Substrate repository. Notable developments include:
- Request parking: when all workers are busy, the router can hold the incoming request and retry resume attempts with bounded wait and caps on how many requests can wait, instead of returning an immediate error. A follow-up change prevents requests to already-running actors from getting stuck behind parked requests.
- Device pluggability via Dynamic Resource Allocation (DRA): device vendors can provide profiles so containers request only the devices they need. This model will work for both gVisor and microVM sandboxes; NVIDIA GPUs are planned as the first supported devices.
- Per-container CPU and memory limits for microVM sandboxes: containers that exceed memory limits are stopped individually so the remainder of the sandbox stays running.
- Preserving GPU state across suspend/resume: GPU checkpoints often require the same GPU model and sometimes the same driver. Agent Substrate proposes that device vendors describe how to pause devices safely and which device properties snapshots depend on; snapshots record those properties and restoration happens only on matching devices, with a fallback to cold start if none are available. The design is vendor-neutral and built on DRA.
Integration with NVIDIA Dynamo: tying agent execution to inference
Agent Substrate manages where agents act; NVIDIA Dynamo, an open-source distributed inference framework, manages where they think. Both layers face the same utilization problem: keeping sandboxes warm wastes CPU/memory, while rebuilding model context wastes GPU cycles. Coordinating them reduces latency and preserves reusable inference state as agents move between tool execution and model calls.
Dynamo’s router tracks where reusable KV cache lives, which GPU workers are busy with prefill or decode, and GPU fleet load. Agent Substrate supplies agent-aware signals—session identity, request priority, expected output length, tool-call boundaries, lifecycle events—so Dynamo can:
- keep related model calls on workers that already have relevant KV cache,
- prioritize interactive turns over background agent work under pressure,
- retain system prompts, tool definitions and conversation history across tool-call gaps,
- drop cache state from terminated subagents or summarized contexts first,
- prepare likely context as a tool call approaches completion.
Crucially, Dynamo returns inference-side signals to Agent Substrate—projected time to first token, projected completion time, request progress, cache readiness, GPU-pool saturation—that inform whether an actor should remain resident, be suspended during a longer inference wait, or be resumed as model responses approach completion. At fleet scale, this feedback helps tune warm-worker capacity, regulate agent fan-out, and avoid activating CPU sandboxes that would sit idle behind saturated GPU queues.
This division of responsibility is simple: Agent Substrate manages high-frequency lifecycle of agent environments; Dynamo manages high-performance lifecycle of inference; the two are connected around the agent loop (model call → tool call → observation → model call).
Deployment and ecosystem fit
Agent Substrate runs above Kubernetes and interoperates with NVIDIA components: KAI Scheduler handles placement, queues, gang scheduling and topology awareness; Dynamo handles model serving and KV-aware routing; DSX OS components provide the validated Kubernetes runtime, GPU operator, and network/topology management. Agent Substrate is an independent CNCF Sandbox project; NVIDIA contributes upstream and develops interoperability with DSX OS, but Substrate itself is not a named DSX OS component.
Getting started and community involvement
A public guide demonstrates deploying a coding assistant using Dynamo and Agent Substrate: running Pi coding agents as Substrate Actors, connecting agents to GLM-5.3-Flash served via NVIDIA Dynamo, and using Harbor to orchestrate evaluations on SWE-bench Verified. The project is still evolving (APIs and GPU support on DRA are in flux; some Dynamo integrations are experimental), and contributions are encouraged via the repository issue trackers and community meetings.
Events (KubeCon + CloudNativeCon North America 2026)
Agent Substrate contributors will be present at KubeCon + CloudNativeCon North America, Nov. 10–12, 2026:
- Agent Substrate Community Party — Tuesday, Nov. 10, Salt Lake City.
- Lightning session — Wednesday, Nov. 11, 13:00–13:15 at Google Cloud Booth #122: “Talk From Pods to Actors: GPU-Aware Infrastructure for Agentic Workloads.”
- NVIDIA booth #170 — Meet the Experts on Nov. 11, 15:15–15:35 MT to discuss Agent Substrate and running agent workloads on Kubernetes.
If you are building agent platforms, coding assistants, or RL environments on Kubernetes, the project asks for workload descriptions, issues, contributions and participation in community meetings to help shape the design going forward.
