Providers

Prefix the model with its provider, as in groq/llama-3.3-70b-versatile or openrouter/anthropic/claude-sonnet-4.6. RouteHub reads the key from api_key= or the first set variable in the key column, and the base URL from api_base= (or base_url=) or the base URL column.

Supported providers

Provider	Model string	API key variables	Base URL variables
OpenAI	`gpt-5.4`, `openai/gpt-4o`	`OPENAI_API_KEY`	`OPENAI_BASE_URL`, `OPENAI_API_BASE`
Anthropic (native API)	`claude-sonnet-4-6`, `anthropic/claude-opus-4-7`	`ANTHROPIC_API_KEY`	`ANTHROPIC_API_BASE`, `ANTHROPIC_BASE_URL`
Google Gemini	`gemini/gemini-3-flash-preview`	`GEMINI_API_KEY`, `GOOGLE_API_KEY`	`GEMINI_API_BASE`
Azure OpenAI	`azure/<deployment>`	`AZURE_API_KEY`, `AZURE_OPENAI_API_KEY`	`AZURE_API_BASE`, `AZURE_OPENAI_ENDPOINT`
Groq	`groq/llama-3.3-70b-versatile`	`GROQ_API_KEY`	`GROQ_API_BASE`
xAI	`xai/grok-4`	`XAI_API_KEY`	`XAI_API_BASE`
DeepSeek	`deepseek/deepseek-chat`	`DEEPSEEK_API_KEY`	`DEEPSEEK_API_BASE`
OpenRouter	`openrouter/anthropic/claude-sonnet-4.6`	`OPENROUTER_API_KEY`, `OR_API_KEY`	`OPENROUTER_API_BASE`
Together AI	`together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo`	`TOGETHERAI_API_KEY`, `TOGETHER_API_KEY`, `TOGETHER_AI_TOKEN`	`TOGETHERAI_API_BASE`
Mistral	`mistral/mistral-large-latest`	`MISTRAL_API_KEY`	`MISTRAL_API_BASE`
Cohere	`cohere/...`	`COHERE_API_KEY`, `CO_API_KEY`	none
Fireworks	`fireworks_ai/accounts/fireworks/models/...`	`FIREWORKS_API_KEY`, `FIREWORKS_AI_API_KEY`	`FIREWORKS_API_BASE`
Perplexity	`perplexity/sonar`	`PERPLEXITYAI_API_KEY`, `PERPLEXITY_API_KEY`	none
Cerebras	`cerebras/...`	`CEREBRAS_API_KEY`	none
DeepInfra	`deepinfra/...`	`DEEPINFRA_API_KEY`	none
SambaNova	`sambanova/...`	`SAMBANOVA_API_KEY`	none
NVIDIA NIM	`nvidia_nim/...`	`NVIDIA_NIM_API_KEY`	`NVIDIA_NIM_API_BASE`
Moonshot	`moonshot/...`	`MOONSHOT_API_KEY`	none
Alibaba DashScope	`dashscope/...`	`DASHSCOPE_API_KEY`	none
Hugging Face	`huggingface/...`	`HF_TOKEN`, `HUGGINGFACE_API_KEY`	none
Ollama	`ollama/llama3`, `ollama_chat/llama3`	none	`OLLAMA_API_BASE`
LM Studio	`lm_studio/...`	optional, `LM_STUDIO_API_KEY`	`LM_STUDIO_API_BASE`
vLLM	`hosted_vllm/...`	optional, `HOSTED_VLLM_API_KEY`	`HOSTED_VLLM_API_BASE`
Any OpenAI-compatible server	`openai_like/<model>`, or any name with `api_base=`	optional, `OPENAI_LIKE_API_KEY`	`OPENAI_LIKE_API_BASE`

Claude is called through Anthropic’s native Messages API rather than its OpenAI-compatible endpoint, which drops prompt caching, thinking output and cached-token usage. RouteHub translates the request and returns a standard ChatCompletion.

Bare model names

These prefixes need no provider:

Prefix	Provider
`gpt-`, `o1`, `o3`, `o4`, `chatgpt-`, `codex-`, `text-embedding-`, `davinci`, `babbage`, `omni-moderation`	OpenAI
`claude-`	Anthropic
`gemini-`, `gemma-`	Gemini
`grok-`	xAI
`deepseek-`	DeepSeek
`mistral-`, `codestral`, `magistral-`, `ministral-`, `pixtral-`, `devstral-`	Mistral
`command-`	Cohere

Any other bare name raises BadRequestError before a request is sent, unless you pass api_base=. Then RouteHub sends it to that server as openai_like. You can also set the provider for an unprefixed name with custom_llm_provider=.

Check a model string

get_llm_provider resolves a model string without sending anything. It returns (model, provider, api_key, api_base):

import routehub

model, provider, api_key, api_base = routehub.get_llm_provider(
    "groq/llama-3.3-70b-versatile"
)
print(model, provider)  # llama-3.3-70b-versatile groq

Provider notes

Azure OpenAI

Azure needs an API version as well as a key and endpoint. Pass api_version=, or set AZURE_API_VERSION or OPENAI_API_VERSION.

response = routehub.completion(
    model="azure/my-gpt-deployment",
    messages=messages,
    api_version="2024-10-21",
)

Local servers: Ollama, LM Studio and vLLM

Ollama defaults to http://localhost:11434/v1 and LM Studio to http://localhost:1234/v1. RouteHub appends /v1 to OLLAMA_API_BASE when it is missing. vLLM and openai_like have no default base URL, so set the variable or pass api_base=.Local servers get a placeholder key when none is set: ollama, lm-studio, or EMPTY for vLLM and openai_like.

routehub.completion(model="ollama/llama3", messages=messages)
routehub.completion(
    model="hosted_vllm/meta-llama/Llama-3.3-70B-Instruct",
    messages=messages,
    api_base="http://gpu-box:8000/v1",
)

Any OpenAI-compatible server

Pass api_base= with any model name to call a server that implements the OpenAI chat API:

routehub.completion(
    model="my-internal-model",
    messages=messages,
    api_base="https://llm.internal.example.com/v1",
    api_key="sk-internal",
)

litellm provider aliases

These litellm prefixes are accepted and mapped: together to together_ai, fireworks to fireworks_ai, nvidia to nvidia_nim, google and google_ai_studio to gemini, x-ai to xai, vllm to hosted_vllm, custom_openai to openai_like, cohere_chat to cohere, and text-completion-openai to openai.

Getting Started

Guides

Reference

Supported providers

Bare model names

Check a model string

Provider notes

​Supported providers

​Bare model names

​Check a model string

​Provider notes

Supported providers

Bare model names

Check a model string

Provider notes