Microservices changed software architecture by breaking monoliths into smaller, independently deployable services. That trade-off improved development and scaling but introduced a new pain: system behavior became harder to observe end-to-end. AI agents are now following a similar path. Workflows that once lived inside one application are being split across multiple agents, each responsible for a specific task. Individual pieces can behave correctly while the overall workflow slowly degrades. The difference today is speed and cost: agents are much faster and cheaper to create, so the same architectural mistakes are easier to repeat at scale.
A three-week investigation from my own design
I learned this the hard way. My name was on the design doc. Splitting that workflow across three agents was a reasonable decision and I still defend most of the reasoning. But that afternoon, if you’d asked me how we would detect the system quietly failing, I had no answer. Months later, during a quality review, a reviewer noticed something felt off before any of our instrumentation did. Latency was fine, error rates were flat, and every service in the chain returned what it was supposed to return. What we had was a slow decline in output quality, and it took about three weeks to trace. The first agent handed a set of documents to the next for summarization. The summarizer dropped a field it judged to be low signal. A downstream agent then made a confident recommendation without that field. Each agent did what it was designed to do, and the result was still wrong. I should have anticipated it — I’d seen this pattern during the microservices years.
The bill we already paid
Breaking monoliths was the right move for most teams, but one thing got forgotten: decomposition didn’t make the hard parts vanish; it moved them into the gaps between services, where the people accountable for the product could no longer see them. The bill arrived around 2019, and it hit delivery schedules rather than incident channels. Teams were running hundreds of services with limited tracing and no contract versioning; the real symptom was slower delivery because engineers spent weeks picking through those services.
Agents are following the same arc faster: standing up an agent can take an afternoon and a prompt instead of a repo and deploy pipeline. When creating an agent is that cheap, product teams make agents without coordination, and nobody keeps track of what agents exist and what they do.
What breaks and what it costs you
This is not only my experience. Cemri and coauthors annotated more than 1,600 execution traces across seven popular multi-agent frameworks and built a taxonomy of 14 recurring failure modes (MAST, NeurIPS 2025). Most failures trace back to how the system was specified and how agents coordinated, rather than to the models themselves.
Inventory sounds trivial but it isn’t. Service sprawl at least showed up in Kubernetes manifests. Agents are created in product surfaces, workflow tools, and someone's weekend automation, and you don’t discover what you have until a compliance reviewer asks which automated system touched a decision — and then it’s too late.
The visibility gap cost us three weeks. When an agent calls an agent that calls a retrieval layer that calls another agent, and every hop contains a model deciding what to pass forward, you need trace context that survives the entire path. Most stacks log prompts and completions per call and stop there: you get each hop but not the chain — and it was the chain that failed.
Compounding confidence worries me most. Distributed services fail loudly, which was a blessing: a service returns a 500 and the caller knows to stop. Agents fail quietly and fluently, so when an upstream agent produces a plausible but wrong summary, the downstream agent receives no signal and reasons confidently on top of it. By the time the result reaches the user, it conveys a level of confidence the evidence doesn’t support.
API contracts drifted when someone changed response formats without versioning them; we built checkers to catch that at build time. Agent contracts drift when someone edits a system prompt — there’s no schema diff, no type check. Nothing breaks until a downstream agent starts getting different output and adapts without telling anyone.
What to carry over from microservices
A registry is the cheapest thing to build and the first thing everyone skips. In the microservices era we kept a service catalog with an owner and scope for each service. For agents we need the same record plus an explicit list of what each agent is allowed to call.
Tracing carries over nearly unchanged: correlation IDs were invented for this problem. OpenTelemetry’s GenAI semantic conventions give you vocabulary, and an agent run becomes a span tree instead of a pile of unrelated log lines. Had we emitted those spans, the summarizer span and the recommender span would have appeared side by side in one trace, and the missing field would have been a diff instead of a three-week hunt. These conventions are still evolving, so keep attribute names behind a thin mapping layer of your own.
Finding the missing field in a trace would have shortened the investigation. I’d also want a test that catches its omission before a prompt change reaches production. Apply contract-testing discipline from microservices: define what a downstream agent depends on, then verify an upstream change preserves it. For the summarizer in our workflow the evaluation must check whether its output retained the information the recommender required. Version prompts and require changes to pass those evaluations before release.
There is early evidence it pays off: SEMAP, a protocol-layer method applying behavioral contract modeling and lifecycle verification on Google’s A2A infrastructure, reported up to roughly 70 percent fewer failures on function-level coding tasks when scored against MAST.
The trust boundary is what I’ll argue about in design reviews. In 2019 we learned a service isn’t trusted just because it lives inside the perimeter — much of what service meshes addressed. The agent equivalent is that an agent shouldn’t get broad tool access just because we built it. What an agent is allowed to do belongs in a policy layer the model can’t talk its way past, not in a polite system prompt.
Cost is where the analogy breaks. A microservice stuck in a retry loop costs CPU and an alert at 3 a.m. An agent stuck in a loop costs real money on every iteration and doesn’t saturate in a way that makes itself obvious. This burden lands on the product owner rather than infrastructure, because it shows up as unit economics rather than an outage; the product manager ends up justifying the number at the end of a billing cycle.
Questions I ask before approving a split
None of this argues against multi-agent systems — they earn their keep like services did. My point is that operational practices must evolve with the architecture, without repeating the five-year lag of the prior transition. The person accountable for delivering the product is usually best placed to ensure that happens.
I push on four things. Can we follow one request all the way through and reconstruct what each agent decided and why? If not, we’re building something we can’t debug. What happens when an upstream agent is wrong — not whether it can be wrong, but how the system responds? If the honest answer is that the next agent carries on as normal, we haven’t designed a failure mode; we designed a failure and left it running.
Who owns this agent, and where is its tool access enforced? A system prompt isn’t enforcement. What does this cost at ten times today’s volume, and what shuts it off when the number goes wrong? I ask that early because nobody volunteers it.
The teams that managed service decomposition most effectively incorporated tracing and contract testing from the start. Those practices helped them understand interactions and verify changes preserved expected behavior. Today that work is a sprint or two; it becomes far larger once you’re running 40 agents and someone asks which of them touched a customer’s data.
