Choosing the Right Smart Routing Strategy for Your Use Case

Smart routing is one of OpenModex's most powerful features, but choosing the right strategy can mean the difference between a 20% cost savings and a 50% one. This guide breaks down each routing strategy with concrete examples so you can make the right choice for your application.

What is Smart Routing?

When you send a request with model: "auto", OpenModex's routing engine evaluates your request against multiple candidate models and selects the best one based on your chosen strategy. The engine considers the request type, token count, required capabilities (vision, function calling, etc.), and real-time provider performance data.

Three built-in strategies cover the vast majority of use cases: cost-optimized, quality-first, and latency-minimized. You can also create custom strategies that blend these priorities.

Strategy 1: Cost-Optimized

The cost-optimized strategy selects the cheapest model that meets a minimum quality threshold for each request type.

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Classify this email as spam or not spam" }],
  routing: {
    strategy: "cost-optimized",
    min_quality: 0.85,        // Minimum acceptable quality score
  },
});

Best for: High-volume, low-complexity tasks where cost is the primary concern. Examples include text classification, sentiment analysis, entity extraction, simple summarization, and content moderation.

How it works: The engine maintains a quality score for each model on each task type, derived from continuous evaluation benchmarks. For a simple classification task, a $0.10/1M-token model might score 0.92 quality while a $3.00/1M-token frontier model scores 0.97. The cost-optimized strategy picks the cheaper model because it exceeds the 0.85 minimum threshold.

Typical savings: 40-60% compared to always using a frontier model.

Strategy 2: Quality-First

The quality-first strategy always selects the model with the highest quality score for the specific request type, regardless of cost.

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Analyze this legal contract for risks..." }],
  routing: {
    strategy: "quality",
    max_cost_per_request: 0.50,  // Optional cost ceiling
  },
});

Best for: High-stakes tasks where accuracy is critical. Examples include legal document analysis, medical information processing, complex code generation, financial modeling, and customer-facing content that represents your brand.

How it works: The engine ranks models by their quality score for the detected task type and selects the top performer. The optional max_cost_per_request parameter lets you set a ceiling so quality optimization does not result in unexpectedly expensive requests.

Typical savings: 10-20% (savings come from avoiding unnecessarily expensive models when a different provider's frontier model is both better and cheaper).

Strategy 3: Latency-Minimized

The latency-minimized strategy selects the model with the fastest current response time, using real-time performance data.

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Quick answer: What year was Python created?" }],
  routing: {
    strategy: "latency",
    max_latency_ms: 500,      // Maximum acceptable latency
    min_quality: 0.80,        // Still enforce a quality floor
  },
});

Best for: Interactive applications where response time directly impacts user experience. Examples include chatbots, autocomplete, real-time translation, and any UI where users are waiting for AI output.

How it works: The engine queries real-time latency data for each candidate model, factoring in current load, geographic proximity, and recent performance trends. It selects the fastest model that meets the quality floor.

Typical savings: 20-35% cost reduction (faster models tend to be cheaper) plus 40-60% latency improvement.

Custom Strategies: Blending Priorities

For most production applications, a blend of priorities is ideal. OpenModex lets you define custom strategies with weighted scoring:

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Generate a product description..." }],
  routing: {
    strategy: "custom",
    weights: {
      cost: 0.4,              // 40% weight on cost
      quality: 0.4,           // 40% weight on quality
      latency: 0.2,           // 20% weight on latency
    },
  },
});

This balanced approach works well for general-purpose applications where you want good quality without overspending or keeping users waiting.

Per-Route Configuration

In real applications, different features have different requirements. A chatbot needs low latency, while a background report generator needs low cost. Configure routing per feature:

// Interactive chat -- optimize for latency
const chatResponse = await client.chat.completions.create({
  model: "auto",
  messages: chatMessages,
  routing: { strategy: "latency" },
});

// Background summarization -- optimize for cost
const summaryResponse = await client.chat.completions.create({
  model: "auto",
  messages: summaryMessages,
  routing: { strategy: "cost-optimized" },
});

// Legal analysis -- optimize for quality
const analysisResponse = await client.chat.completions.create({
  model: "auto",
  messages: analysisMessages,
  routing: { strategy: "quality" },
});

Monitoring Routing Decisions

The OpenModex dashboard shows which models were selected for each request and why. This transparency lets you validate that routing decisions align with your expectations and tune your strategies over time.

Every API response includes a x-openmodex-model header telling you which model was actually used, so you can log and analyze routing decisions in your own systems.

Getting Started

Start with cost-optimized routing at a 0.90 quality threshold. This is the safest default -- it saves money on easy tasks while preserving quality where it matters. Monitor the dashboard for a week, then adjust thresholds based on your specific quality requirements.