Sign in
rate-limits

Create Rate Limit Rule

Changed on

Add a rule. It applies from the next request on this replica, and on every replica within 30 seconds.

post/api/v1/rate-limits

Request

  • The document declares no server URL.
  • Auth: one of:
    • API key in header Otari-Key
    • API key in header x-api-key

Request body

lease_secnumber

How long a max_concurrent slot is held at most. A slot is given back when its response ends; this bounds what a process that dies mid-request keeps.

max_concurrentinteger nullable

Requests in flight at once.

modelsstring[] nullable

The models a per: model rule limits, each as instance:model (the provider instance the model is called through, then the model), each counted on its own. Required there and refused on any other rule.

namestring required

Names the rule in a 429's detail and in the counter's key. Unique across rate_limits.

per'deployment' | 'key' | 'user' | 'model' required

What one count is shared by.

rpminteger nullable

Requests per minute.

tpminteger nullable

Tokens per minute, counted on what each request used; see tpm_admission for how a request is admitted.

tpm_admission'used' | 'estimate'

How a tpm limit admits a request. 'used' (the default) admits a request while the minute's tokens are under the limit and counts what it used once it completes, as LiteLLM does. 'estimate' holds the request's estimate (prompt plus max output, or budget_estimate_default_output_tokens) while it runs and refuses it when that does not fit, which is how providers count their own quotas: use it for a limit meant to stay under one.

Example request

{
  "name": "keys",
  "per": "key",
  "rpm": 600,
  "tpm": 200000
}

Response

Successful Response

lease_secnumber

How long a max_concurrent slot is held at most. A slot is given back when its response ends; this bounds what a process that dies mid-request keeps.

max_concurrentinteger nullable

Requests in flight at once.

modelsstring[] nullable

The models a per: model rule limits, each as instance:model (the provider instance the model is called through, then the model), each counted on its own. Required there and refused on any other rule.

namestring required

Names the rule in a 429's detail and in the counter's key. Unique across rate_limits.

per'deployment' | 'key' | 'user' | 'model' required

What one count is shared by.

rpminteger nullable

Requests per minute.

source'config' | 'dashboard' required

'config' for a rule from config.yml, which is read-only here; 'dashboard' for a stored one.

tpminteger nullable

Tokens per minute, counted on what each request used; see tpm_admission for how a request is admitted.

tpm_admission'used' | 'estimate'

How a tpm limit admits a request. 'used' (the default) admits a request while the minute's tokens are under the limit and counts what it used once it completes, as LiteLLM does. 'estimate' holds the request's estimate (prompt plus max output, or budget_estimate_default_output_tokens) while it runs and refuses it when that does not fit, which is how providers count their own quotas: use it for a limit meant to stay under one.

updated_atstring date-time nullable

When a stored rule last changed.

Changes

  • fbee488d4d9fRevision changes
    • info

      added the optional property to the response with the status

  • e99152573057Revision changes
    • info

      added the new optional request property

    • info

      added the optional property to the response with the status