Researchers at Alibaba developed SkillWeaver, a framework designed to address the problem of routing subtasks to the right tools in large enterprise AI agent environments. The central issue is that agents may have hundreds or thousands of modular “skills” (tool specifications) and naively exposing the entire library to a language model overwhelms context windows, wastes tokens, and often leads to incorrect tool choices.
How SkillWeaver works
SkillWeaver structures the process into three stages: Decompose, Retrieve, and Compose.
- Decompose: a task-decomposer LLM (in experiments, Qwen2.5-7B-Instruct) breaks a complex user request into a sequence of atomic sub-tasks.
- Retrieve: an embedding-based retriever (baseline all-MiniLM-L6-v2 with FAISS) fetches a shortlist of candidate skills for each subtask from the skill library.
- Compose: a planner evaluates candidate combinations for inter-skill compatibility and assembles a final execution plan represented as a Directed Acyclic Graph (DAG), allowing independent nodes to run in parallel when possible.
The goal is to enable agents to autonomously orchestrate multi-tool workflows common in real-world business operations (e.g., download dataset → transform data → generate visual reports).
Skill-Aware Decomposition (SAD)
The framework’s key innovation is Iterative Skill-Aware Decomposition (SAD), a feedback loop that aligns the LLM’s decomposition with the actual vocabulary and granularity of available skills. The loop works by having the LLM draft an initial decomposition, performing a preliminary retrieval to find loosely matching skills, and then feeding those retrieved skill hints back into the LLM so it can rewrite the decomposition. This retrieve-and-rewrite cycle produces step descriptions that better match real tools and improves downstream retrieval and planning.
Experimental setup and results
The authors evaluated SkillWeaver on a custom benchmark called CompSkillBench containing 300 multi-step queries and used a library of 2,209 real-world skills drawn from the public Model Context Protocol (MCP) ecosystem across 24 functional categories (cloud, finance, databases, etc.). Key findings:
- The 7B model in the vanilla (no-SAD) setup achieved decomposition accuracy of 51.0%; enabling SAD raised this to 67.7%.
- With a larger Qwen-Max model and SAD, decomposition accuracy reached 92% in their measurements.
- On hard tasks requiring four to five distinct skills, SAD improved accuracy by 50%.
- A larger 14B model in the vanilla setup sometimes performed worse than the 7B model due to over-decomposition; adding SAD anchored the model and improved results.
- A ReAct-style baseline completely failed on decomposition (0% accuracy).
Token usage and practical impact:
- The LLM-Direct baseline (feeding all tools into a large model) required an estimated ~884,000 tokens of context per query. SkillWeaver’s targeted retrieve-and-route approach reduced context consumption to roughly 1,160 tokens per query — a ~99.9% reduction.
- This reduction implies much lower API costs and faster response times in real deployments.
Embedding and retrieval observations:
- The core retrieval used all-MiniLM-L6-v2 embeddings, and switching to BGE-base-en-v1.5 improved accuracy without fine-tuning.
- An off-the-shelf bi-encoder can place a relevant tool into the top-10 candidates about 70% of the time, but only ranks the exact best tool first roughly 37% of the time; a secondary cross-encoder or LLM-based reranker is likely needed to reorder the top candidates reliably.
- Building a FAISS index for all 2,209 skills took about 15 seconds in the experiments, and retrieval adds under 15 ms latency per query.
Limitations and developer considerations
The authors have not released SkillWeaver’s source code, but the approach is reproducible with off-the-shelf components (LangChain, LlamaIndex, or plain Python). Practical notes for implementers:
- SAD is essentially a prompt-engineering plus retrieval loop; the paper includes prompt templates that teams can adapt.
- Vectorizing and indexing the tool library is a required upfront step, but the paper reports it as a negligible cost in practice.
- SkillWeaver as presented focuses on routing and planning; it does not include built-in error recovery, retries, or fallback logic. In production, developers must add retry/fallback/compensation mechanisms to handle API failures or malformed outputs in multi-step chains.
Conclusion
SkillWeaver demonstrates that compositional routing with an iterative, skill-aware feedback loop substantially improves multi-step decomposition accuracy while cutting token usage dramatically. The results suggest that aligning an agent’s decomposition to the concrete vocabulary of available tools can be more effective than solely increasing LLM size, and that retrieve-and-rewrite approaches offer strong practical benefits for enterprise multi-tool orchestration. However, production deployments will need additional engineering for robustness and error handling.
