Modal, the AI infrastructure platform led by Erik Bernhardsson, aims to make running code in the cloud truly serverless: developers should not have to provision clusters, predict load, or pay for idle GPUs. In recent months the company reported rapid growth: it says it crossed a $1 billion annualized run rate, closed a $750 million Series D at a $16 billion valuation, and has spun up nearly three billion sandboxes.
The problem: demand isn’t rectangular
A typical capacity contract gives a fixed number of GPUs for a fixed period; plotted over time this looks like a rectangle. Real workloads, however, spike, grow and shrink. Buying for peak capacity means paying for unused hardware much of the time; buying less risks underprovisioning.
Modal’s strategy, as Bernhardsson explains it, is a duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer draw what they need when they need it. The smoother the aggregated demand becomes, the more efficiently the pool can be used.
This contrasts with providers focused on making capacity tradeable; Modal’s emphasis is on slicing capacity into smaller, elastic pieces that are immediately usable.
Financial trade‑offs: locked capacity versus price risk
Modal has historically used spot and short reservations from hyperscalers (AWS, GCP) but the market has shifted. To get the capacity they now need, Modal works with nearly 25 clouds and often signs multi‑year (three‑year) deals with upfront payments. That approach reduces short‑term availability risk but requires capital and exposes the business to price risk: if GPU prices later fall, a company holding expensive, long‑term commitments may suffer losses.
Bernhardsson says he does not expect an imminent price collapse but stresses that the risk of “buying at the top” is a real consideration. At the same time, constrained supply can raise prices in the short run, which can benefit providers who have reservations locked in.
Where demand came from: the sandbox story
Modal’s core product initially was serverless functions, which fit inference workloads and autoscaling. Sandboxes were developed when the team realized LLMs frequently generate code and users needed a place to run it. Early adoption was modest and the product was nearly shut down, but the growth of coding agents, background agents and reinforcement‑learning workloads dramatically increased sandbox demand.
Today sandboxes contribute roughly a third of Modal’s revenue, a proportion the leadership did not predict at launch. The rise of agentic workflows also changed infrastructure needs: agents repeatedly create, inspect and mutate environments, so startup and restore latency for sandboxes became part of the product experience.
Fast startup as an infrastructure feature
Elasticity only works if capacity can be provisioned fast enough to matter. Modal’s techniques include a buffer of available GPUs, a custom filesystem, process checkpointing and GPU memory snapshots. These mechanisms reduce the work required to bring a model replica online and directly affect responsiveness and cost.
How Modal obtains capacity
As the company grew it moved from mostly spot reservations to using a wider array of providers and longer commitments. Modal rents rather than owns hardware so far; to operate at scale it built a procurement desk and a team that manages bare‑metal capacity worldwide, dealing with network issues, failing drives, and offline GPUs.
Bernhardsson emphasizes the trade‑offs: renting avoids some operational complexity of owning hardware (vendors, power, cables) and the financing questions (debt vs equity) that come with purchases, but buying can also make sense under some circumstances. Modal says it is open to combinations of renting and buying depending on economics and risk.
Alternatives to NVIDIA and neocloud dynamics
Modal watches non‑NVIDIA accelerators (TPUs, Cerebras, Groq) as options that may gain traction over time. While NVIDIA remains dominant, alternative hardware can be attractive for particular workloads and may grow as software support improves.
Neoclouds have shown they can scale quickly using long‑term purchase commitments as collateral to raise financing. Modal has taken a different, product‑led path, focusing on developer experience and on‑demand access, which changes its financial profile versus neocloud players.
Product mix and future bets
Modal identifies two main products: Modal Functions (serverless functions, heavily used for inference) and Modal Sandboxes. Functions make up about two‑thirds of revenue; sandboxes about one‑third.
The company is also investing in training (especially smaller and mid‑sized training jobs), batch workloads (large‑scale offline processing), low‑latency inference, and agent‑centric primitives. They’re working on features such as VM support, enhanced snapshotting, live migration and forking to lower the overhead of creating and destroying environments.
Security and the role of agents
A public incident discussed in the conversation involved an app running on Modal that was exploited; the issue was an application vulnerability, not a platform compromise. Modal has been considering stricter default security settings and has begun using agents internally to probe for sandbox escapes — employing agents both offensively (to find vulnerabilities) and defensively.
Forward‑deployed engineers and product evolution
Modal has expanded its cadre of forward‑deployed engineers (FDEs) who help customers with performance tuning, migrations and optimizations. Part of their job is to identify repetitive, high‑value work that should be converted into product features so that customers won’t require bespoke assistance indefinitely.
On engineering culture, Bernhardsson notes that Modal treats code review as more optional when strong CI/CD and testing exist; the company emphasizes automation, observability and fault‑isolation to keep operations reliable.
What it all means
- The rectangular nature of typical GPU contracts mismatches most real workloads; Modal’s value proposition is to aggregate and smooth demand so capacity can be sliced and shared elastically.
- Infrastructure choices (snapshotting, checkpointing, custom filesystems) are not just implementation details: they change latency, cost, and which applications are feasible.
- Financially, long‑term reservations reduce short‑term scarcity but require capital and carry price‑risk; Modal currently rents most capacity under multi‑year deals with partners but keeps buying decisions on the table.
Modal positions itself as a platform providing both a developer experience to speed iteration and a capacity‑aggregation layer that can reallocate compute quickly across models and users. That dual focus helps explain the company’s rapid growth and its confidence that it can translate bursty, agent‑driven demand into a stable, scalable service.
Key metrics mentioned
- Reported > $1 billion annualized run rate.
- $750 million Series D at a $16 billion valuation.
- Nearly three billion sandboxes created.
Those milestones, together with Modal’s infrastructure investments, are central to the company’s thesis that GPU‑market frictions (fixed rectangular supply commitments) can be reinterpreted into flexible, on‑demand slices for developers and agents.
