Model routing is increasingly central to enterprise AI stacks: it routes incoming prompts to the best-suited model to optimize latency and cost. Most existing systems, however, treat routing as a static classification task, which limits their effectiveness.
A new open-source framework called Agent-as-a-Router reframes the router as a dynamic agent that builds memory. The approach relies on a Context-Action-Feedback (C-A-F) loop to record model successes and failures and to adapt routing behavior based on execution-grounded information.
The researchers also published ACRouter, a concrete implementation of this paradigm. In evaluations, ACRouter significantly outperformed static routers and the expensive strategy of always defaulting to frontier models — without requiring teams to train very large models or write extensive hand-crafted heuristics.
The economics of routing and the information deficit
Single-model deployments can be useful for experiments but are problematic at scale. Model routing lets engineers map tasks to cheaper, faster open models when appropriate, while reserving costly frontier models for complex reasoning.
Today, developers rely on two main mechanisms: (1) heuristics-based routing using hand-coded rules (for example, if a prompt contains specific keywords route to GPT-5.5, otherwise send to a self-hosted open model like Kimi K2.7); and (2) static trained policies or classifiers that predict the best model based on prompt embeddings and historical training data.
Both approaches are static. When the researchers tested these mechanisms on real-world coding and agentic workflows, they found a hard accuracy ceiling. Static routers suffer from a severe information deficit: they only evaluate input text and never observe whether the chosen model actually succeeded, so they effectively guess on complex edge cases.
This leads to three failure modes: (a) frozen information state — inability to accumulate execution feedback in deployment; (b) poor out-of-distribution (OOD) generalization — breakdown when enterprise data or user behavior shifts; and (c) vulnerability to model churn — a classifier trained on today’s models can become obsolete when better models appear.
Agent-as-a-Router: a self-evolving system
Agent-as-a-Router argues that an effective router must acquire and accumulate execution-grounded information during deployment — in other words, learn on the job.
The C-A-F loop works as follows: when a new prompt arrives, the router inspects the prompt and task metadata (e.g., programming language or difficulty). It searches its memory for similar past tasks and which models succeeded or failed. Using that context, it selects a target model and executes the task. The system then observes the real-world outcome, extracts a success/failure signal, and writes that feedback back into memory to inform future routing.
For example, in an automated enterprise analytics pipeline the router might send a SQL generation task to an open model like Kimi. If the model hallucinates a column name and the SQL fails to compile, the C-A-F loop observes the compiler error, registers the failure, and logs it. Next time a similar obscure query arrives, the router may route it to a stronger model such as Claude Opus 4.8.
ACRouter architecture
ACRouter implements the framework with three core components: Orchestrator, Verifier, and Memory, supported by a tool layer that executes the C-A-F loop in the real world.
- Memory: built on a vector store, it retrieves relevant past interactions and updates the historical database with new outcomes, powering the context phase.
- Orchestrator: handles the action phase, processing the user prompt and retrieved memory to select the most capable target model from the available pool.
- Verifier: manages the feedback phase by evaluating the chosen model’s output and generating a clear success/failure signal.
The tool layer connects the Verifier to execution environments (a Python interpreter, an agentic sandbox, or a database engine), allowing the system to run generated code or queries and observe exact outcomes.
The Orchestrator is intentionally lightweight: rather than a huge LLM, the researchers trained a sub-billion-parameter adapter based on Qwen 3.5 (about 0.8B parameters), making self-hosting feasible.
ACRouter in action: benchmark results
To evaluate the approach, the researchers introduced CodeRouterBench, an environment of roughly 10,000 tasks with verified scores across eight frontier models, including Claude Opus 4.6, GPT-5.4, Qwen3-Max, and GLM-5. The evaluation included in-distribution (ID) tests across nine single-turn coding dimensions (e.g., algorithm design, test generation) and out-of-distribution (OOD) agentic programming tests that required multi-step planning, file navigation, and iterative debugging.
Baselines showed no single model dominates every category. For example, although Claude Opus 4.6 had the highest average performance, GLM-5 outperformed it in algorithm design with an 86% relative improvement, and Qwen3-Max beat it in test generation with a 111% improvement — while Opus cost roughly 12× more than smaller models like Kimi-K2.5.
Static routers repeatedly failed by routing niche coding tasks to ill-suited models and not seeing that execution failed. ACRouter, by contrast, adjusted after receiving execution feedback.
According to the benchmarking, ACRouter occupies the Pareto frontier of cost and performance. On both ID and OOD tests it achieved the lowest cumulative regret (a metric of sub-optimal routing decisions over time). On the in-distribution test set, ACRouter’s full task run cost $13.21, compared to $34.02 for always defaulting to Opus — about a 2.6× cost saving.
ACRouter dynamically matched tasks to the most capable model for each niche, suggesting enterprises can reach or surpass frontier-level accuracy across diverse workloads without paying premium prices for every request.
Caveats, limitations, and how to get started
Agent-as-a-Router addresses the information deficit but is not a universal solution.
The framework is best for verifiable tasks where the Verifier receives a clear success/failure signal from the environment (e.g., coding, data retrieval), and for domains with distribution shifts or where different models excel in distinct niches.
It is overkill for trivial tasks where any model suffices, or for low-volume applications where engineering overhead is not justified. It is also unsuitable for subjective domains (such as creative writing) where correct answers are not easily verifiable and feedback signals cannot be standardized.
The researchers open-sourced the code on GitHub and released the Orchestrator model weights on Hugging Face under the Apache 2.0 license. The router is compatible with Claude Code, Codex, and OpenCode.
Summary
Agent-as-a-Router and its ACRouter implementation demonstrate that routing that accumulates execution-grounded feedback can significantly improve model selection while reducing costs. The approach is particularly applicable to verifiable, multi-model enterprise workflows where continuous learning from real outcomes addresses distribution shifts and model churn.
