Industry

AI-generated text

Measuring the Cost of a User Story in an Agentic Code Factory

Between June 25 and July 18, 2026, an autonomous SDLC framework built hundreds of story deliveries while recording detailed usage for one repository run.

Measuring the Cost of a User Story in an Agentic Code Factory

Between March and July 2026 François‑Xavier Martin used an autonomous SDLC framework to produce an 861,601‑line application with Claude Code. The first generation of that framework did not persist usage data, and Claude Code’s default 30‑day transcript retention erased the remaining records, so the original bill is lost.

The analysis in this article comes from a later, instrumented second‑generation factory: the claude‑code‑config framework records every stage’s token usage (input, output, cache read, cache write), cost, model, and failure category into a ledger as it runs. The dataset covers every story built in one repository between June 25 and July 18, 2026: 17 runs, 193 story‑builds, 374 stage attempts and 336 session logs.

June runs used Claude Opus 4.8, July runs used Claude Fable 5, and smaller pieces were handled by Claude Haiku 4.5.

Defining a story and a story‑build

In this pipeline a “story” is an agile requirement decomposed from an epic with acceptance criteria that provide the agent a stopping condition. A “story‑build” is that story’s full automated delivery cycle: tests‑first, build, coverage gate, review by a dedicated reviewer agent, and merge, including bugfixes and re‑asks when an agent’s reply is malformed. Up to five stories ran concurrently in isolated git worktrees.

What a story cost (the ledger and the logs)

The factory consumed 595.7 million tokens to ship 77 stories — 7.7 million tokens per delivered story. At the list prices for the models used, that equates to $837.53 total or $10.88 per story. The numerator counts every token thrown away, the five FAILED stories, 22 failed stage attempts, bugfix and re‑ask loops, and retries; the denominator counts only stories that shipped.

Earlier rough estimates placed story costs between a few dollars and a few tens of dollars; measured results here range from $3.02 to $43.24 with a median of $9.56, so the earlier range was reasonable. The most expensive story cost $43.24 (a 3‑point story that had a review retry and a bugfix loop). The wall‑clock mean is about 2.5× the median because one overnight run stalled twice on the subscription plan’s rate‑limit window and waited hours — a billing artifact rather than an agent one.

The ghost application mentioned at the start had 696 stories; at this measured rate that would imply roughly 5.4 billion tokens, which cannot be verified because the original logs are gone.

The factory is mostly a reading machine

A key finding: 95.4% of tokens were cache reads. The factory re‑reads roughly 73 cached tokens for every new token it writes or receives. The API charges much less for a cache read (about a tenth of fresh input) but charges a premium to write new content into the cache.

Although cache writes are only 3.3% of tokens, they account for 31.2% of the bill; overall cache traffic drives 77% of cost. Fresh input is only about 1.6% of the cost.

This distribution was consistent across three independent samples — production runs, interactive framework‑development sessions, and fragments from the ghost application — two framework generations and two working modes. It appears to be a structural property of how agentic development consumes compute.

Practical consequence: cost optimization in an agentic pipeline is primarily cache management, not prompt shortening. Context discipline, cache‑tier awareness and orchestrators that avoid stuffing their own windows move the bill; trimming prompt wording does not.

The honest denominator: failures and rework

There are two ways to count failures. The narrow view — attempts explicitly marked FAILED — represents 5.0% of tokens. The honest view, which includes all rework (retries, bugfix and re‑ask loops) plus crashed sessions that streamed tokens and then died, is about 13%. Public cost claims rarely clarify which reading they use.

Only 34 of 76 measured stories were clean first‑pass, but rework stays relatively cheap because retries are small compared to full builds. The author treats the 13% as a quality bill because gates catch problems.

A meter that lied and the need for audit

While dissecting raw data the author found a bug: the ledger missed about one‑sixth of real consumption, recording $694.65 versus the logs’ $837.53. When a result envelope failed validation the controller’s re‑ask overwrote the original stage row’s usage, erasing the expensive failed session from the books; crashed sessions never wrote back at all. Fifty‑seven attempts were affected, which is why the session logs are the ground truth.

The meter needed auditing. The author filed the bug against his own factory; the report decomposed into three defects and repaired the overwrite and model recording in one merged PR (issue #480, PR #482, 3,200 tests passing). The factory audited and fixed most of its own meter; work to recover spend from crashed sessions remains open.

Who actually pays

Marginally, the bill was zero: the author runs a $200/month Max 20x subscription, so every dollar in the article is labeled API‑equivalent. The subscription’s real currency is quota, not money: the overnight run stalled twice on the five‑hour rate‑limit window, and ten dispatches waited 3.3–4.2 hours before auto‑resuming. On a flat monthly fee, time is the constraint.

One rolling month of measured work across all three codebases totaled about $1,088 API‑equivalent versus the $200 fee — more than five to one — and that’s a floor because older transcripts are purged. This demonstrates an asymmetry between list rates and the subscription experience: list price is not necessarily the provider’s cost; it includes margin.

Nothing in the plan terms explicitly prevents a professional or small firm from running on flat fees; the distinction Anthropic draws is contractual (consumer vs Team/business terms), not strictly financial. A Team premium seat at $125 buys business terms and central administration but roughly half the quota per dollar. Quotas and tiers can tighten and reprice; a factory that meters itself will notice the day the economics change.

What the meter changes

The author discovered that all measurements were produced with model‑routing switched off. Mechanical merges burned premium‑model prices on Haiku‑grade work, which comprised 12.3% of tokens. In other words, 7.7 million tokens per delivered story is the unoptimized rate; fixing model routing is in the factory backlog.

A second finding: writing the factory’s specifications — epics and stories in interactive sessions — consumed roughly 190 million tokens, about 25 stories’ worth (~$160 in converted terms). When implementation is this cheap, the code itself is no longer the single expensive artifact. The difference between the $10.88 measured story and the unknowable 861,601 lines is that one pipeline recorded its bill.

Summary

  • In agentic code factories, cache reads dominate token volume and cost; cache management matters far more than prompt length for cost optimization.
  • Metering must be correct and audited: measurement bugs can erase substantial consumption from the books.
  • Subscription quota structures create a practical divergence from list prices; flat fees buy time/quota and governance, not necessarily tokens.

The claude‑code‑config factory and the local‑code‑bench repository are public; methodology, CSVs and extraction scripts accompany the analysis so readers can verify the numbers. The production application referenced at the start remains private, so its costs remain unknown.