Drew Breunig, CEO and co-founder of cmpnd.ai, introduced the term “prompt debt” at Friends of O’Reilly (Foo Camp) and later discussed it on Live with Tim O’Reilly. He defines prompt debt as the hidden cost teams accumulate when they fight a model’s trained behavior instead of working with it: numerous small, compensatory rules and edge-case patches get embedded in prompts to counteract default model behavior. Over time these hacks break or conflict with model upgrades, producing a technical-debt-like burden.
Practical effects of prompt debt
Breunig lists three main costs:
- Slowed iteration: prompts become cluttered with repeating fixes; adding a new instruction risks regressions, which discourages changes.
- Collaboration blockages: a prompt edited by someone else can look arbitrary and fragile, so others avoid touching it even if it’s suboptimal.
- Lock-in to specific models: many hacks are tuned to one set of weights. Breunig cites Datadog’s State of AI Engineering report, which found GPT-4o was still the most common model in Datadog request traces in March 2026 despite its retirement from the ChatGPT UI. Teams often keep 18-month-old to two-year-old models in production because they dread rebuilding prompts.
Datadog also reported that 69% of input tokens in customer traces were system prompts rather than user content, underscoring how heavily systems rely on prompts.
Why prompt debt arises
Breunig gives two causes:
-
Natural language imprecision: framing and phrasing matter. The same intent phrased differently can produce different results. He showed studies where a patient’s request to taper alprazolam is refused by assistants, while a psychiatrist asking about the same case with professional phrasing gets the protocol. Breunig also referenced research showing strange sensitivities (e.g., declaring allegiance to the Philadelphia Eagles influenced a model’s willingness to explain illegal plant import methods).
-
Fighting the weights: models embody their developers’ preferences. Breunig and Srihari Sriraman analyzed system prompts for six major coding agents and found the same instructions repeated five or seven times in a single prompt, escalating from IMPORTANT to CRITICAL to MANDATORY and even threatened penalties — essentially “war-driving the thesaurus” until some phrasing sticks.
The harness being trained into the model and architectural consequences
Breunig tracked Claude Code’s published system prompts and observed they shorten after a model release and then grow again. Anthropic patches unreliable behavior via prompts and then bakes those patches into the next model, which helps Claude Code but complicates custom harnesses: models may assume they’re running inside Claude Code and call Claude Code tool APIs, forcing other harnesses (e.g., Pi) to repeatedly tell the model otherwise.
Breunig describes three choices for those fighting the weights: solve it in your prompt; catch and retry in the harness; or make your API look like what the model expects. Steve Yegge’s hack—adding aliases for model-expected call names—works but lets model expectations shape everyone’s software.
Tim O’Reilly draws a parallel to the early web: while some vendors folded features into their servers, Apache kept a clean extension layer and that modular architecture enabled broad innovation. Breunig worries big labs may be making a strategic mistake by integrating harness behavior into models: it improves predictable tasks and creates a moat, but risks closing off outside innovation. He characterizes labs as “cornered rather than greedy” because the empty-textbox interface must work both for large harness builders and for inexperienced users, forcing strong defaults.
The cost of trading diversity for reliability
If you don’t give a model detailed instructions, you get back the average of everything it has seen — a path toward monoculture, Drew warns. He gives the example of image generation: café flyers in New York and Mumbai can look the same because models converge to shiny, identical outputs. Optimizing for reliability reduces surprise.
Breunig argues that heavy post-training focused on verifiable tasks (coding, math, tool use) drowns out the human signal from pre-training; more post-training can degrade creative capabilities. He notes that Fable and GPT-5 write worse than earlier models on creative metrics, and he doubts a single model can excel at both very reliable code and richly diverse prose.
How to address prompt debt
During audience Q&A, Breunig offered practical guidance:
- Time limits? No fixed timeframe; instead teach teams to recognize the “prompt debt smell”: repeated instructions, one-off edge-case patches, and increasingly desperate wording. Those cues indicate logic should move into evals and automation.
- Quantitative measures: track how often prompts change, how many people edited them, which prompts have been untouched for a year, and which are editable by only one person. If you’re forced to run on old models and can’t migrate, that’s a strong sign of prompt debt.
- Fastest-compounding habits: “vibe shipping” — quick prototyping, piling patches into prompts, and shipping without building maintainable systems. Each patch is an eval inside the prompt that vanishes when you change models.
- Pseudocode and DSPy: sometimes models optimize toward pseudocode. Breunig highlighted DSPy and its Flex optimizer as tools that avoid pushing logic into prompts: simple cases are handled in code and the LLM is called only when needed. His advice: treat prompts as perishable, define tasks by measurements not paragraphs, and automate prompt discovery for your target model so you can swap in cheaper, faster, or newer models without starting over.
- Multi-agent workflows: helpful for decomposition—split tasks into smaller, evaluable steps rather than one giant prompt. That improves cost, reliability, governance, and speed.
- Compliance needs: decompose tasks into stages with checkpoints so you can inspect how the model reached a result instead of trusting a single opaque end-to-end answer.
Breunig acknowledged trade-offs in frameworks like DSPy: any abstraction gives up some flexibility, but DSPy aims to keep the task-spec layer stable while letting the implementation evolve.
Final thought: it’s our job to make models "weird"
Breunig is optimistic: if a model’s default output is the average of everything it has seen, humans must push it out of its distribution — “make it weird.” Practically, that means understanding model training biases and compensating for them. Breunig gave examples: deliberately avoiding React for a frontend because models are heavily trained on React and thus produce homogenized results; choosing GLM and Kimi not for cost but for malleability inside a custom harness.
He supports keeping an open-weight ecosystem so models remain infrastructure rather than turning into closed appliances. Open weights preserve the modularity and participation that drive innovation.
Summary
Prompt debt describes how incremental prompt hacks to fight a model’s tendencies slow development, block collaboration, and tether systems to older models. Breunig recommends moving logic out of perishable prompts into evals, measurable task specs, decomposition, multi-agent workflows and toolchains (like DSPy) so systems can upgrade models and preserve diversity and external innovation.
