Skip to main content

Overview

The ModelRouter is an intelligent routing system that automatically selects and executes AI models based on task requirements. It leverages a function-calling architecture to analyze tasks and recommend the optimal model and provider combination for each specific use case.

Key Features

  • Dynamic model selection based on task complexity and requirements
  • Multi-provider support (OpenAI, Anthropic, Google, etc.)
  • Concurrent and asynchronous execution capabilities
  • Batch processing with memory
  • Automatic error handling and retries
  • Provider-aware routing
  • Cost optimization

Installation

  1. Install the latest version of swarms:
  1. Set up your API keys in your .env file:

Attributes

str
default:"model_router_system_prompt"
Custom prompt for guiding model selection behavior.
int
default:"4000"
Maximum token limit for model outputs.
float
default:"0.5"
Control parameter for response randomness (0.0-1.0).
int | str
default:"10"
Maximum concurrent workers. Use "auto" for CPU count.
str
default:"None"
API key for model access.
int
default:"1"
Maximum number of refinement iterations.

Methods

step()

Runs a single routing step: selects a model/provider for the task, executes it, and returns the output. run() calls this internally in a loop up to max_loops times.
Parameters:
  • task (str): The task to be executed
Returns: str - The result of the single routing step

run()

Executes a single task through the model router with memory and refinement capabilities.
Parameters:
  • task (str): The task to be executed
Returns: str - The result of task execution

batch_run()

Executes multiple tasks sequentially with result aggregation.
Parameters:
  • tasks (list): List of task strings to be executed
Returns: list - List of results, one for each task

concurrent_run()

Parallel execution of multiple tasks using thread pooling.
Parameters:
  • tasks (list): List of task strings to be executed
Returns: list - List of results from parallel execution

async_run()

Asynchronous task execution with coroutine support.
Parameters:
  • task (str): The task to be executed
Returns: asyncio.Task - An asyncio Task object

Usage Examples

Basic Usage

Batch Processing

Concurrent Execution

Asynchronous Execution

Financial Analysis System

Healthcare Data Processing Pipeline

NLP Processing Pipeline

Available Models and Use Cases

Provider Capabilities

Performance Optimization Tips

  1. Token Management
    • Set appropriate max_tokens based on task complexity
    • Monitor token usage for cost optimization
    • Use streaming for long outputs
  2. Concurrency Settings
    • Adjust max_workers based on system resources
    • Use "auto" workers for optimal CPU utilization
    • Monitor memory usage with large batch sizes
  3. Temperature Tuning
    • Lower (0.1-0.3) for factual/analytical tasks
    • Higher (0.7-0.9) for creative tasks
    • Mid-range (0.4-0.6) for balanced outputs
  4. System Prompts
    • Customize for specific domains
    • Include relevant context
    • Define clear output formats

Dependencies

  • asyncio: Asynchronous I/O support
  • concurrent.futures: Thread pool execution
  • pydantic: Data validation
  • litellm: LLM interface standardization

Source Code

View the source code on GitHub