ssl_verify=False or retry policy never leaks into another’s when many teams or agents share a process.
Gateway options
int
default:"2"
Retries on rate limits, server errors, timeouts and connection failures, with exponential backoff that honours
retry-after. None means 2.bool
default:"False"
Drop what the model rejects instead of failing: sampling parameters on OpenAI reasoning models,
reasoning_effort on models without reasoning, thinking on older Claude models, sampling on Claude 4.7 and later, and forced tool_choice on Claude 5.1 and later. Unknown keyword arguments are then not sent either.bool | str
default:"True"
TLS verification, or a path to a corporate CA bundle.
float
default:"600.0"
Request timeout in seconds.
timeout= wins when set.float
default:"60.0"
Seconds an idle pooled connection stays open, so agent steps that arrive a few seconds apart reuse the same connection.
bool
default:"None"
Gzip the request body sent to Claude’s Messages API, which saves upload time on long histories. OpenAI-compatible providers always get an uncompressed body.
None reads ROUTEHUB_GZIP_ON.bool
default:"False"
Print the provider, model, parameter names and timing to stderr. API keys and message content are never printed.
OpenAI | AsyncOpenAI | httpx.Client
default:"None"
Your own configured client:
openai.OpenAI or AsyncOpenAI for OpenAI-compatible providers, an httpx Client or AsyncClient for Claude. No API key is required, and the client’s own TLS, pool and retry settings apply.str | Exception
default:"None"
Return a canned response, or raise the given exception, without a network call. See Testing.
Provider options
str
The provider’s API key. Defaults to the provider’s environment variable, read on each call.
str
Base URL for a proxy, private endpoint or self-hosted server.
base_url= is an alias.str
API version, for Azure OpenAI.
str
The provider for a model string without a prefix.
dict
Extra HTTP headers for the request.
dict
Extra fields merged into the request body. Always sent, even with
drop_params.stop, seed, n, logprobs or prompt_cache_key, can be passed by name. Unrecognized keyword arguments are sent in the request body unless drop_params is on.