Introducing Smart Routing v2: ML-Powered Model Selection
Our new Smart Routing v2 engine uses machine learning to automatically select the best AI model for every request. Learn how it works and how it can reduce your costs by up to 40% while maintaining quality.
Sarah Kim
CTO & Co-Founder
Today we are excited to announce Smart Routing v2, a major upgrade to our intelligent model selection engine. This new version uses machine learning to analyze your requests in real-time and route them to the optimal AI model based on your specific requirements for cost, latency, and quality.
The original Smart Routing was rule-based -- you could set simple conditions like "use the cheapest model for summarization tasks" or "prefer low-latency models for chat." While useful, this approach required manual configuration and did not adapt to changing model performance over time.
Smart Routing v2 changes everything. Our ML engine continuously learns from millions of API requests across the OpenModex platform, building a deep understanding of which models perform best for different types of tasks, input lengths, and quality requirements.
How It Works
Here is how it works under the hood: When you send a request with model="auto", our routing engine first classifies the request type (chat, analysis, coding, creative, etc.) and complexity. It then queries our performance database for real-time latency, availability, and quality scores for each candidate model. Finally, it applies your routing strategy (cost, latency, or quality optimized) to select the best model.
Benchmark Results
In our internal benchmarks, Smart Routing v2 reduces costs by an average of 37% compared to always using a single model, while maintaining a 95th percentile quality match. For latency-optimized routing, we see median response times decrease by 45% as the engine learns to avoid temporarily slow providers.
Getting Started
Smart Routing v2 is available today for all Pro and Enterprise customers. Free tier users can try it with up to 1,000 routed requests per month. To get started, simply set model="auto" in your API requests and configure your preferred routing strategy in the dashboard.
New Analytics
We are also releasing a new Analytics page that shows you exactly how Smart Routing is performing -- which models were selected, estimated cost savings, and quality comparisons. This transparency is core to our philosophy: you should always know what is happening with your AI requests.
What's Next
Looking ahead, we are working on custom routing rules that let you combine automated ML selection with business-specific constraints. For example, you might require that all requests from European users are routed to EU-hosted models, or that certain sensitive topics always use a specific model. Stay tuned for more updates.