Adaptive Model Routing Cuts Multi-Agent LLM Inference Costs by Up to 90%

The proposed architecture shifts planning from a static, upfront global planner to a just-in-time (JIT) process where each agent generates its own sub-tasks. This design mitigates the "context blindness" of traditional orchestrators, allowing downstream agents to make routing decisions based on actual data gathered by upstream steps. The system uses a simple scoring mechanism (0-6) to map tasks to model tiers, ensuring that simple data retrieval uses low-cost models while complex analysis utilizes powerful ones.

Experimental results show that this adaptive strategy can cut costs by up to 90% for straightforward queries without compromising quality on complex tasks. The solution is provider-agnostic and implemented via a FastAPI backend with real-time cost tracking. This method addresses the financial inefficiency of running all agent steps on frontier models, offering a scalable path for enterprise AI deployments to balance performance with budget constraints.