# Routing and caching

One model can have several hosts: companies that serve it, each with its own price and terms. A call goes to the first suitable host and, if that one failed through its own fault (5xx, timeout, `429`), to the next. The call is billed at the price of the host that answered, plus our 20%. A model's hosts, their prices and terms come from `GET /v1/models` in `endpoints`, and the host directory from `GET /v1/providers` (both without a key).

Within one price, the calls of one conversation go to the host that answered the previous call: it already holds the [cache](#prompt-caching). The choice is kept for an hour without calls.

## Choosing hosts: provider

The `provider` object in the request body chooses hosts the way OpenRouter's does. It is never passed to the host.

| Key | Description |
| --- | --- |
| `only` | The hosts the call may go to, by name (a model's `endpoints` in `GET /v1/models`); every other host is excluded. |
| `ignore` | The hosts the call must not go to. |
| `order` | The hosts tried first, in this order; the others follow unless `allow_fallbacks` is `false`. |
| `allow_fallbacks` | `false` keeps the call on the hosts named in `order`. Default `true`. |
| `sort` | `price` puts the cheapest host first. `throughput` and `latency` are refused with `unsupported_provider_preference`: no host's speed is measured yet. The object form `{"by": "price", "partition": "model"}` is read the same way. |
| `max_price` | A price ceiling per host, in US dollars per 1M tokens: `prompt` and `completion`. A host priced above it is excluded; none left is `price_above_max`. `request` and `image` are accepted and not applied: a call has no per-request or per-image charge here. |
| `require_parameters` | Always `true` here, whatever is sent: a host that does not take a field of the request (`tools`) is never chosen. |
| `data_collection` | `deny` excludes the hosts that may train on the data or do not declare their terms. Default `allow`. |
| `zdr` | `true` keeps the call on hosts with signed zero-retention terms. |
| `data_region` | Keeps the call on hosts that process the text in this region (`ru`). A model with no such host is refused, never sent abroad as a fallback. |

`max_price` is in US dollars per 1M tokens, as on OpenRouter: code written for OpenRouter works unchanged. It is compared with the price you pay, that is, the host's price already including our 20%. The balance is charged in roubles at that price, converted at the official Central Bank of Russia rate of the day of the call. A `provider` key the table does not list is refused, not skipped: a promise about data we do not keep is worse than a refusal.

| HTTP | type | Code | message | Retry |
| --- | --- | --- | --- | --- |
| 400 | `invalid_request_error` | `invalid_provider_preferences` | provider must be an object of OpenRouter's provider preferences. | no |
| 400 | `invalid_request_error` | `unsupported_provider_preference` | The provider object has a preference this API does not honour. | no |
| 400 | `invalid_request_error` | `unknown_provider` | The provider object names a host this API does not route to. | no |
| 400 | `invalid_request_error` | `price_above_max` | Every eligible host of the model is priced above provider.max_price. | no |
| 404 | `not_found_error` | `no_eligible_host` | No host of the model meets the request's provider preferences. | no |

The answer to `no_eligible_host` and `price_above_max` names why each host was excluded.

## Fallback models: models

`models: [...]` sets a chain of models, as on OpenRouter. If no host of the first model answered through its own fault, or the prompt did not fit its window, the call moves on to the next one. The `model` field, when present, comes first in the chain. The answer names the model that answered and is billed at its host. The chain's length is capped; extra entries and non-strings are refused with the code `invalid_models`.

```json
{
  "model": "<vendor>/<model>",
  "models": ["<vendor>/<fallback-model>"],
  "messages": [{"role": "user", "content": "…"}]
}
```

## Model choice auto

`model: "auto"` picks the model itself: the cheapest for this call among those that keep a host after your `provider` filters (data terms and `max_price` included), have tools when the request has tools, and whose window holds the prompt and the asked output. The choice is pinned to the conversation for an hour without calls so the cache is kept; the model changes only when it stops fitting (the prompt outgrew its window). The answer names the model that answered.

The `auto-router` plugin narrows the choice with `allowed_models`, as on OpenRouter: `"plugins": [{"id": "auto-router", "allowed_models": ["<vendor>/*"]}]` (a `*` is allowed in a pattern).

If no model fits, the call is refused with the code `no_auto_model`.

## Data in Russia

Some models are served by hosts that process the request text on their own servers in Russia (for example Yandex AI Studio and GigaChat). On the [Models](/models) page such a model carries the mark "Data in Russia", and the page links to a section with the prices of hosts in Russia only while the catalogue has at least one such model. Each model page marks whether each host processes data in Russia. The page names only where a host processes data and does not replace your own 152-FZ compliance assessment.

- `GET /v1/models?data_region=ru` returns only the models that have a host in Russia, with only those hosts in `endpoints`.
- With `"provider": {"data_region": "ru"}` in the request, the call goes only to hosts in Russia. If the model has no such host, or none of them answered, the call is refused: it never moves on to a host abroad. A `models` chain and `auto` read the same filter for every model.
- The call is billed at the price of the host in Russia that answered, plus our 20%, like any other call.

```json
{
  "model": "<vendor>/<model>",
  "messages": [{"role": "user", "content": "…"}],
  "provider": {"data_region": "ru"}
}
```

## Retention and training

`"provider": {"zdr": true}` keeps the call on hosts with signed zero-retention terms, and `"data_collection": "deny"` excludes hosts that may train on the data or do not declare their terms. Each host's terms and links to its privacy policy and terms of service come from `GET /v1/providers`. We store neither the prompt nor the answer: the call log keeps the model, the host, the tokens, the cost and how the call ended.

## Prompt caching

Put the repeating part (system, tool definitions, the early messages of the conversation) first and keep it unchanged between calls, and the next call reads it from the cache at a lower price. How the cache is kept depends on the host, named in `endpoints[].cache`: `auto_prefix` hosts cache the longest repeated start of the prompt by themselves, there is nothing to pass.

To keep the calls of one conversation in one cache, name the conversation. The key is read in this order: the `x-claude-code-session-id` header (Claude Code sends it), `session_id`, `prompt_cache_key`, `metadata.user_id` on the Anthropic wire, `user`. The key stays inside your organization: hosts receive a key salted with your organization, never the one you sent. Without a key the conversation is told by its system prompt and its first message.

On `/v1/messages`, `cache_control` marks are kept: a host that speaks Anthropic Messages itself (`messages` in `endpoints[].wires`) receives them in the body; a Chat Completions host that caches at marks (`explicit_breakpoints`) gets them carried onto its message parts; on the other hosts they have nothing to bind to and the host caches on its own. Of the mark itself only `type` goes through:

| Key | What happens to it |
| --- | --- |
| `type` | passed to the host as sent |
| `ttl` | accepted and dropped |
| `scope` | accepted and dropped |

`ttl` and `scope` are removed because they change a price the call's hold does not account for: the mark stays and lives for the host's default time (five minutes at Anthropic). There is no one-hour cache through this API. The cache is already partitioned by organization, so `scope` is not needed.

Claude Code puts a line `x-anthropic-billing-header: …` with a value that changes on every request first in the system prompt. It breaks any prefix cache, so we remove it before the prompt reaches the host.

The answer says how much was read from and written into the cache:

| Wire | Read | Written |
| --- | --- | --- |
| OpenAI | `usage.prompt_tokens_details.cached_tokens` | `usage.prompt_tokens_details.cache_write_tokens` |
| Anthropic | `usage.cache_read_input_tokens` | `usage.cache_creation_input_tokens` (split by tier in `usage.cache_creation`) |

Cache reads are paid at the `input_cache_read` price, cache writes at `input_cache_write`, of the host that answered. The cache saving in roubles is `cache_discount` in [`GET /v1/generation`](/docs/model-api-requests#accounting-and-cost).

## Plugins

`plugins` takes the OpenRouter plugins the table lists; any other is refused with the code `unsupported_plugin`.

| Plugin id | What happens to it |
| --- | --- |
| `auto-router` | replaced: read by this API, the host gets this API's value |
| `context-compression` | accepted and dropped |
