auto router

Get Auto Router Benchmarks

Benchmarks for the auto-router dashboard: session shape, savings against the configured baseline, and prompt-caching behaviour bucketed by what the router did.

Reads session rollups folded once per request at spend-write time, so this endpoint never scans LiteLLM_SpendLogs. A user filter selects only turns attributed to that internal user when written; older key-only history remains outside user views. A session is in the window when it overlaps it: its last turn is on or after start_date and its first turn is on or before end_date. Overall hit rate is over telemetry-bearing turns; each bucket's hit rate is over that bucket's turns.

The rollup supplies the measures, never the list. Which routers appear comes from the model registry, so one shows up as soon as it is configured and reads zero until it serves traffic, and routers_in_scope counts those too rather than only the routers the window recorded.

get/auto_router/benchmarks

Request

  • Base URL: https://docs.litellm.com.cn (the document declares no server; this is the origin it was published from)
  • URL: https://docs.litellm.com.cn/auto_router/benchmarks
  • Auth: API key in header x-litellm-api-key

Query parameters

start_datestring nullable

YYYY-MM-DD UTC, inclusive (defaults to 30 days before end_date)

end_datestring nullable

YYYY-MM-DD UTC, inclusive (defaults to today)

api_keystring nullable

Filter to one virtual key token hash

user_idstring nullable

Filter to one canonical internal user recorded on each turn

Response

Successful Response

start_datestring required

Window start day, YYYY-MM-DD UTC, inclusive

end_datestring required

Window end day, YYYY-MM-DD UTC, inclusive

routers_in_scopeinteger required

How many groups this response carries. Every auto-router configured on the proxy counts, whether or not it served anything in the window. To count only the routers that did serve traffic, filter groups to the entries whose sessions is above zero

Changes

No recorded changes to this operation across all 1 revision of this API.