Skip to main content

Overview

The ModelRouter is an intelligent routing system that automatically selects and executes AI models based on task requirements. It leverages a function-calling architecture to analyze tasks and recommend the optimal model and provider combination for each specific use case.

Key Features

  • Dynamic model selection based on task complexity and requirements
  • Multi-provider support (OpenAI, Anthropic, Google, etc.)
  • Concurrent and asynchronous execution capabilities
  • Batch processing with memory
  • Automatic error handling and retries
  • Provider-aware routing
  • Cost optimization

Installation

  1. Install the latest version of swarms:
  1. Set up your API keys in your .env file:

Attributes

str
default:"model_router_system_prompt"
Custom prompt for guiding model selection behavior.
int
default:"4000"
Maximum token limit for model outputs.
float
default:"0.5"
Control parameter for response randomness (0.0-1.0).
int | str
default:"10"
Maximum concurrent workers. Use "auto" for CPU count.
str
default:"None"
API key for model access.
int
default:"1"
Maximum number of refinement iterations.

Methods

step()

Runs a single routing step: selects a model/provider for the task, executes it, and returns the output. run() calls this internally in a loop up to max_loops times.
Parameters:
  • task (str): The task to be executed
Returns: str - The result of the single routing step

run()

Executes a single task through the model router with memory and refinement capabilities.
Parameters:
  • task (str): The task to be executed
Returns: str - The result of task execution

batch_run()

Executes multiple tasks sequentially with result aggregation.
Parameters:
  • tasks (list): List of task strings to be executed
Returns: list - List of results, one for each task

concurrent_run()

Parallel execution of multiple tasks using thread pooling.
Parameters:
  • tasks (list): List of task strings to be executed
Returns: list - List of results from parallel execution

async_run()

async_run() is currently broken and should not be used. Its implementation is return asyncio.create_task(self.run(task, *args, **kwargs)), but run() is a plain sync def — so self.run(...) executes eagerly (blocking the event loop) and returns a plain str before asyncio.create_task() ever sees it. Passing that string to create_task() raises TypeError: a coroutine was expected, which async_run() catches and re-raises as RuntimeError: Async execution failed: .... Every await router.async_run(...) call site fails. Use the synchronous run() (optionally wrapped in asyncio.to_thread() from an async context) or concurrent_run() / batch_run() for parallel execution instead.
Documented (broken) signature:
Working alternative from an async context:

Usage Examples

Basic Usage

Batch Processing

Concurrent Execution

Asynchronous Execution

async_run() is currently broken (see the Warning above). To call ModelRouter from an async context without blocking the event loop, offload the synchronous run() to a thread with asyncio.to_thread():

Financial Analysis System

Healthcare Data Processing Pipeline

NLP Processing Pipeline

Available Models and Use Cases

Provider Capabilities

Performance Optimization Tips

  1. Token Management
    • Set appropriate max_tokens based on task complexity
    • Monitor token usage for cost optimization
    • Use streaming for long outputs
  2. Concurrency Settings
    • Adjust max_workers based on system resources
    • Use "auto" workers for optimal CPU utilization
    • Monitor memory usage with large batch sizes
  3. Temperature Tuning
    • Lower (0.1-0.3) for factual/analytical tasks
    • Higher (0.7-0.9) for creative tasks
    • Mid-range (0.4-0.6) for balanced outputs
  4. System Prompts
    • Customize for specific domains
    • Include relevant context
    • Define clear output formats

Dependencies

  • asyncio: Asynchronous I/O support
  • concurrent.futures: Thread pool execution
  • pydantic: Data validation
  • litellm: LLM interface standardization

Source Code

View the source code on GitHub