DeepSeek’s recent decision to cut prices on its V4‑Pro model by 75% would seem like an unambiguous win for enterprise AI vendors and developers. In practice, however, cheaper models are not automatically improving margins. The core issue is straightforward: inference unit costs are falling, but agent systems are consuming tokens far faster than prices decline.
For two decades software economics followed a clear rule: infrastructure got cheaper each year while applications became more capable. Many expected AI to follow the same trajectory—frontier models improve and token prices drop until inference becomes a negligible operating expense. That assumption is now breaking down.
What token amplification is
A simple chatbot typically converts one user question into roughly one model call. An agent converts the same question into a chain of planning, retrieval, tool use, verification, summarization, and follow‑up decisions. The user sees one answer; the vendor pays for the whole loop. This is the 100x problem: the same user‑visible request can cost many times more to serve as an agentic workflow than as a chatbot or RAG response. Longer workflows raise the multiplier further.
Falling model prices help, but they do not fix a product architecture that turns one prompt into dozens of billable operations.
Concrete numbers and an example
In a single‑turn chatbot the input‑to‑billed ratio is roughly 1:5. In a multi‑step agent deployed across customer support, sales ops, finance, legal review, and engineering, that ratio routinely lands at 1:700 or higher. Every loop iteration carries forward cumulative conversation, tool outputs, and reasoning traces — nothing is dropped automatically.
A “simple” agent query like “What did our top customer ask about last week?” typically touches seven priced operations before returning an answer:
- User prompt (~50 tokens)
- System prompt and tool definitions (~3,000 tokens, repeated on every call)
- Retrieval (~5,000 tokens of context)
- Model call #1 — tool selection (8,000 in / 200 out)
- Tool execution (~4,000 tokens returned)
- Model call #2 — summarization (12,000 in / 400 out)
- Model call #3 — follow‑up decision (12,400 in / 100 out)
One sentence in can translate to roughly 35,000 input tokens billed. On frontier models that can cost somewhere between $0.10 and $0.40 per query. Multiply that by a million queries a month — a baseline volume for enterprise B2B features — and the monthly line item reaches six figures.
Why this breaks the existing AI business model
The dominant enterprise AI pricing story has been seat‑based SaaS: charge per user per month, deliver agent capability, capture margin. That model assumes a reasonably bounded cost‑per‑user. Token amplification breaks that assumption. A power user running 50 agent invocations a day on a $40/seat plan can generate more inference cost than the plan charges. When that happens, vendor gross margins go negative.
Several vendors now report privately that heavy users produce negative gross margins, mirroring cloud expenditure reports where AI‑agent adoption is correlated with margin contraction. Public coverage has started to reflect these strains; Bloomberg documented a widening gap between Salesforce’s Agentforce demos and the capabilities shipping to customers — the predictable result when promised functionality is technically feasible but uneconomical at the implied seat price.
As Nvidia’s Bryan Catanzaro put it: “For my team, the cost of compute is far beyond the costs of the employees.” The strategic implication isn’t merely that AI is expensive; it’s that the dominant business model assumed by many AI‑native plans does not survive contact with agentic workloads.
Agent orchestration as the new moat
The technical responses are known and converging. They are not novel, but they are critical for survival:
- Cost‑aware routing: a small classifier model decides which tier handles each query. Well‑tuned routers can cut inference bills by around 60% without quality loss.
- Prompt caching: Anthropic, OpenAI, and Google offer 75–90% discounts on cached prefixes.
- Context discipline: truncate tool outputs, prune reasoning traces, and cap tool depth to avoid rabbit holes.
- Speculative decoding: for self‑hosted deployments, this can deliver 2–3× effective throughput on the same GPUs.
IBM reports that organizations using orchestration‑led governance see much stronger productivity gains than those relying on compliance‑only approaches. Companies that build this orchestration layer well start to resemble financial trading systems: every routing decision priced, every path with its own P&L, and every tenant on a metered budget.
What enterprise leaders should do
Four moves separate companies that will retain margins in 24 months from those that won’t:
- Make inference cost a first‑class metric. Track it per feature, per tenant, per query class as teams tracked cloud costs a decade ago.
- Budget like a media buyer. Set cost‑per‑thousand‑queries ceilings per feature, cap them, and alert on overruns.
- Treat the router as core infrastructure, not an optimization. It is the new load balancer.
- Audit prompts quarterly. A 4,000‑token system prompt that grew organically over months is a slow‑motion six‑figure bill; many teams have never read their production prompts end‑to‑end.
Additionally, negotiate volume commits early: frontier model vendors now offer reserved‑instance‑style prepaid discounts, and list price is often the worst deal an enterprise will pay.
The next 24 months
The structural shift beneath agentic AI is not that inference unit costs aren’t falling — they are. DeepSeek’s price cut highlights that frontier inference unit costs can drop roughly 3× per year and the curve isn’t slowing. The shift is that amplification is outrunning the price cuts. Cutting per‑token costs by 75% does not help a company whose agents are doing 700× more tokens per user query than its pricing model assumed.
For the first time since the cloud era began, architecture decisions are financial decisions in real time: a prompt redesign is a margin event, and a poorly bound agent loop is an outage with a credit‑card bill attached. The companies that survive the next two years will not be those running the cheapest model but those whose agents are smart and know what they cost to think.
Maitreyi Chatterjee is a senior software engineer at a big tech company. Devansh Agarwal works as an ML engineer at a leading tech company.
