A recent paper examined 67 frontier language models from 21 providers, including GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro. The authors identify a mathematical limit to multi-model orchestration they call the co-failure ceiling: the proportion of prompts where every model in the pool yields an incorrect result. Because this all-wrong rate sets an absolute upper bound on accuracy, expensive routing, voting, or cascade systems can fail to improve—and can sometimes worsen—effective performance while adding latency and operational complexity.
Orchestration architectures and their hidden costs
The study analyzes three common orchestration patterns:
- Model routers: traffic controllers that route complex queries to expensive models and simple ones to cheaper models.
- Cascades: always query a cheap model first and escalate to premium models only when the first model signals low confidence.
- Mixture-of-Agents (MoA): ask multiple models the same question and synthesize an aggregated answer.
All these architectures introduce a shadow price: added latency, more complex infrastructure and governance across multiple API providers, and higher maintenance costs. Teams typically pick models by pairwise error correlation—choosing models that fail on different prompts so they appear complementary.
Why pairwise correlation can be misleading
The paper shows that pairwise error-correlation misses the common-mode failures shared by all models. For example, on the open-ended MATH-500 benchmark the statistical prediction from pairwise correlations estimated a simultaneous wipeout of only 2.3% of questions, but the measured co-failure rate was 5.2%. Standard correlation metrics therefore underestimated the true failure tail by roughly 2.25×.
Josef Chen, the paper’s author, told VentureBeat that naive majority voting across unequal models produced negative mean gains: “diverse-but-weaker members outvote the strong one.” The practical recommendation is to combine only models within a matched quality band; otherwise teams should prefer a single best model rather than assembling unequal members.
Drivers of co-failure and format sensitivity
The researchers found that task format affects the all-wrong tail substantially. Converting graduate-level science questions from the GPQA benchmark from multiple-choice to free-response increased the all-wrong tail to 12.7%. That highlights how open-ended generation inflates the co-failure ceiling.
They distinguish two limiting regimes:
- Ceiling-bound environments (e.g., open-ended math): high co-failure rates mean the task is too difficult and all models fail together; routing cannot fix underlying capability limits.
- Realizability-bound environments (e.g., graduate-level science): co-failure is near zero so at least one model usually knows the answer, but disagreement among models is subtle and a router cannot reliably pick the correct output without an oracle.
A free pre-deployment sanity check: the Clopper–Pearson bound
The authors recommend a zero-cost statistical check using the Clopper–Pearson bound to compute a worst-case, guaranteed upper bound on the system’s achievable accuracy from a small held-out sample. For instance, if a team tests five agents on 50 sample queries and observes that all agents fail together on two questions, a naive reading might claim 96% accuracy. The Clopper–Pearson calculation corrects for sampling uncertainty and might show the true co-failure rate could be as high as 12%.
Practically, enterprises should assemble a held-out dataset (for example, a fintech company could use 200 complex customer support tickets with human-written resolutions as the benchmark). Integration is straightforward: it’s a counting job over existing eval logs, so it can run in the CI stage and be recomputed whenever the model pool or workload changes.
When to buy a single model vs. build an ensemble
The study concludes that for tasks where answers can be definitively checked—such as an executable SQL query, exact invoice extraction from a 50-page PDF, or producing JSON that must match a strict schema—paying for the single best frontier model generally beats stitching together several cheaper models unless the team has an exceptionally strong query-level routing signal.
MoA-positive cases do exist: when you can match model quality, building a diverse ensemble with low pairwise correlation can beat a high-correlation Self-MoA setup (re-querying the same premium model multiple times). But if models are unequal, majority voting can decrease performance.
Engineering implications and final recommendations
The co-failure ceiling is the practical limiter for multi-model strategies: market-leading models often agree, and they tend to fail on the same hard prompts. Because the Clopper–Pearson check is free, any engineering team can track its co-failure rate across model generations to see whether the tail is closing. In short, the lever buyers actually have is failure-mode heterogeneity and market churn—not merely increasing the number of models in a pool.
