Routing and caching
One model can have several hosts: companies that serve it, each with its own price and terms. A call goes to the first suitable host and, if that one failed through its own fault (5xx, timeout, 429), to the next. The call is billed at the price of the host that answered, plus our 20%. A model's hosts, their prices and terms come from GET /v1/models in endpoints, and the host directory from GET /v1/providers (both without a key).
Within one price, the calls of one conversation go to the host that answered the previous call: it already holds the cache. The choice is kept for an hour without calls.
Choosing hosts: provider
The provider object in the request body chooses hosts the way OpenRouter's does. It is never passed to the host.
| Key | Description |
|---|---|
only | The hosts the call may go to, by name (a model's endpoints in GET /v1/models); every other host is excluded. |
ignore | The hosts the call must not go to. |
order | The hosts tried first, in this order; the others follow unless allow_fallbacks is false. |
allow_fallbacks | false keeps the call on the hosts named in order. Default true. |
sort | price puts the cheapest host first. throughput and latency are refused with unsupported_provider_preference: no host's speed is measured yet. The object form {"by": "price", "partition": "model"} is read the same way. |
max_price | A price ceiling per host, in US dollars per 1M tokens: prompt and completion. A host priced above it is excluded; none left is price_above_max. request and image are accepted and not applied: a call has no per-request or per-image charge here. |
require_parameters | Always true here, whatever is sent: a host that does not take a field of the request (tools) is never chosen. |
data_collection | deny excludes the hosts that may train on the data or do not declare their terms. Default allow. |
zdr | true keeps the call on hosts with signed zero-retention terms. |
data_region | Keeps the call on hosts that process the text in this region (ru). A model with no such host is refused, never sent abroad as a fallback. |
max_price is in US dollars per 1M tokens, as on OpenRouter: code written for OpenRouter works unchanged. It is compared with the price you pay, that is, the host's price already including our 20%. The balance is charged in roubles at that price, converted at the official Central Bank of Russia rate of the day of the call. A provider key the table does not list is refused, not skipped: a promise about data we do not keep is worse than a refusal.
| HTTP | type | Code | message | Retry |
|---|---|---|---|---|
| 400 | invalid_request_error | invalid_provider_preferences | provider must be an object of OpenRouter's provider preferences. | no |
| 400 | invalid_request_error | unsupported_provider_preference | The provider object has a preference this API does not honour. | no |
| 400 | invalid_request_error | unknown_provider | The provider object names a host this API does not route to. | no |
| 400 | invalid_request_error | price_above_max | Every eligible host of the model is priced above provider.max_price. | no |
| 404 | not_found_error | no_eligible_host | No host of the model meets the request's provider preferences. | no |
The answer to no_eligible_host and price_above_max names why each host was excluded.
Fallback models: models
models: [...] sets a chain of models, as on OpenRouter. If no host of the first model answered through its own fault, or the prompt did not fit its window, the call moves on to the next one. The model field, when present, comes first in the chain. The answer names the model that answered and is billed at its host. The chain's length is capped; extra entries and non-strings are refused with the code invalid_models.
Model choice auto
model: "auto" picks the model itself: the cheapest for this call among those that keep a host after your provider filters (data terms and max_price included), have tools when the request has tools, and whose window holds the prompt and the asked output. The choice is pinned to the conversation for an hour without calls so the cache is kept; the model changes only when it stops fitting (the prompt outgrew its window). The answer names the model that answered.
The auto-router plugin narrows the choice with allowed_models, as on OpenRouter: "plugins": [{"id": "auto-router", "allowed_models": ["<vendor>/*"]}] (a * is allowed in a pattern).
If no model fits, the call is refused with the code no_auto_model.
Data in Russia
Some models are served by hosts that process the request text on their own servers in Russia (for example Yandex AI Studio and GigaChat). On the Models page such a model carries the mark "Data in Russia", and the page links to a section with the prices of hosts in Russia only while the catalogue has at least one such model. Each model page marks whether each host processes data in Russia. The page names only where a host processes data and does not replace your own 152-FZ compliance assessment.
GET /v1/models?data_region=rureturns only the models that have a host in Russia, with only those hosts inendpoints.- With
"provider": {"data_region": "ru"}in the request, the call goes only to hosts in Russia. If the model has no such host, or none of them answered, the call is refused: it never moves on to a host abroad. Amodelschain andautoread the same filter for every model. - The call is billed at the price of the host in Russia that answered, plus our 20%, like any other call.
Retention and training
"provider": {"zdr": true} keeps the call on hosts with signed zero-retention terms, and "data_collection": "deny" excludes hosts that may train on the data or do not declare their terms. Each host's terms and links to its privacy policy and terms of service come from GET /v1/providers. We store neither the prompt nor the answer: the call log keeps the model, the host, the tokens, the cost and how the call ended.
Prompt caching
Put the repeating part (system, tool definitions, the early messages of the conversation) first and keep it unchanged between calls, and the next call reads it from the cache at a lower price. How the cache is kept depends on the host, named in endpoints[].cache: auto_prefix hosts cache the longest repeated start of the prompt by themselves, there is nothing to pass.
To keep the calls of one conversation in one cache, name the conversation. The key is read in this order: the x-claude-code-session-id header (Claude Code sends it), session_id, prompt_cache_key, metadata.user_id on the Anthropic wire, user. The key stays inside your organization: hosts receive a key salted with your organization, never the one you sent. Without a key the conversation is told by its system prompt and its first message.
On /v1/messages, cache_control marks are kept: a host that speaks Anthropic Messages itself (messages in endpoints[].wires) receives them in the body; a Chat Completions host that caches at marks (explicit_breakpoints) gets them carried onto its message parts; on the other hosts they have nothing to bind to and the host caches on its own. Of the mark itself only type goes through:
| Key | What happens to it |
|---|---|
type | passed to the host as sent |
ttl | accepted and dropped |
scope | accepted and dropped |
ttl and scope are removed because they change a price the call's hold does not account for: the mark stays and lives for the host's default time (five minutes at Anthropic). There is no one-hour cache through this API. The cache is already partitioned by organization, so scope is not needed.
Claude Code puts a line x-anthropic-billing-header: … with a value that changes on every request first in the system prompt. It breaks any prefix cache, so we remove it before the prompt reaches the host.
The answer says how much was read from and written into the cache:
| Wire | Read | Written |
|---|---|---|
| OpenAI | usage.prompt_tokens_details.cached_tokens | usage.prompt_tokens_details.cache_write_tokens |
| Anthropic | usage.cache_read_input_tokens | usage.cache_creation_input_tokens (split by tier in usage.cache_creation) |
Cache reads are paid at the input_cache_read price, cache writes at input_cache_write, of the host that answered. The cache saving in roubles is cache_discount in GET /v1/generation.
Plugins
plugins takes the OpenRouter plugins the table lists; any other is refused with the code unsupported_plugin.
| Plugin id | What happens to it |
|---|---|
auto-router | replaced: read by this API, the host gets this API's value |
context-compression | accepted and dropped |
Frequently asked questions
What currency is provider.max_price in?
US dollars per 1M tokens, as on OpenRouter, so code written for OpenRouter works unchanged. It is compared with the price you pay, that is, already including our 20%. The balance is charged in roubles at the price of the host that answered plus our 20%: a dollar price is converted at the official Central Bank of Russia rate of the day of the call.
How do I keep data processing in Russia?
Pass "provider": {"data_region": "ru"}. The call goes only to hosts that process the text in Russia; if there are none or none of them answered, the call is refused and never moves on to a host abroad.
Does cache_control with ttl "1h" work?
The cache_control mark is kept, and the ttl and scope keys are removed from it: the cache lives for the host's default time. A one-hour cache costs more at the host, and the call's hold does not account for it.