Overview
TheModelRouter is an intelligent routing system that automatically selects and executes AI models based on task requirements. It leverages a function-calling architecture to analyze tasks and recommend the optimal model and provider combination for each specific use case.
Key Features
- Dynamic model selection based on task complexity and requirements
- Multi-provider support (OpenAI, Anthropic, Google, etc.)
- Concurrent and asynchronous execution capabilities
- Batch processing with memory
- Automatic error handling and retries
- Provider-aware routing
- Cost optimization
Installation
- Install the latest version of swarms:
- Set up your API keys in your
.envfile:
Attributes
str
default:"model_router_system_prompt"
Custom prompt for guiding model selection behavior.
int
default:"4000"
Maximum token limit for model outputs.
float
default:"0.5"
Control parameter for response randomness (0.0-1.0).
int | str
default:"10"
Maximum concurrent workers. Use
"auto" for CPU count.str
default:"None"
API key for model access.
int
default:"1"
Maximum number of refinement iterations.
Methods
step()
Runs a single routing step: selects a model/provider for the task, executes it, and returns the output.run() calls this internally in a loop up to max_loops times.
task(str): The task to be executed
str - The result of the single routing step
run()
Executes a single task through the model router with memory and refinement capabilities.task(str): The task to be executed
str - The result of task execution
batch_run()
Executes multiple tasks sequentially with result aggregation.tasks(list): List of task strings to be executed
list - List of results, one for each task
concurrent_run()
Parallel execution of multiple tasks using thread pooling.tasks(list): List of task strings to be executed
list - List of results from parallel execution
async_run()
Documented (broken) signature:Usage Examples
Basic Usage
Batch Processing
Concurrent Execution
Asynchronous Execution
async_run() is currently broken (see the Warning above). To call ModelRouter from an async context without blocking the event loop, offload the synchronous run() to a thread with asyncio.to_thread():
Financial Analysis System
Healthcare Data Processing Pipeline
NLP Processing Pipeline
Available Models and Use Cases
Provider Capabilities
Performance Optimization Tips
-
Token Management
- Set appropriate
max_tokensbased on task complexity - Monitor token usage for cost optimization
- Use streaming for long outputs
- Set appropriate
-
Concurrency Settings
- Adjust
max_workersbased on system resources - Use
"auto"workers for optimal CPU utilization - Monitor memory usage with large batch sizes
- Adjust
-
Temperature Tuning
- Lower (0.1-0.3) for factual/analytical tasks
- Higher (0.7-0.9) for creative tasks
- Mid-range (0.4-0.6) for balanced outputs
-
System Prompts
- Customize for specific domains
- Include relevant context
- Define clear output formats
Dependencies
asyncio: Asynchronous I/O supportconcurrent.futures: Thread pool executionpydantic: Data validationlitellm: LLM interface standardization