auto router

Get Auto Router Benchmarks

Changed on

Benchmarks for the auto-router dashboard: session shape, savings against the configured baseline, and prompt-caching behaviour bucketed by what the router did.

Reads session rollups folded once per request at spend-write time, so this endpoint never scans LiteLLM_SpendLogs. A user filter selects only turns attributed to that internal user when written; older key-only history remains outside user views. Money counts only requests on the selected UTC days, and the all-router savings headline is the same daily total the Overall view reads. Session shape and caching cover every session that overlaps the window, whole. Overall hit rate is over telemetry-bearing turns; each bucket's hit rate is over that bucket's turns.

The rollup supplies the measures, never the list. Which routers appear comes from the model registry, so one shows up as soon as it is configured and reads zero until it serves traffic, and routers_in_scope counts those too rather than only the routers the window recorded.

get/auto_router/benchmarks

Request

  • Base URL: https://api.sea-lion.ai (the document declares no server; this is the origin it was published from)
  • URL: https://api.sea-lion.ai/auto_router/benchmarks
  • Auth: API key in header x-litellm-api-key

Query parameters

start_datestring nullable

YYYY-MM-DD UTC, inclusive (defaults to 30 days before end_date)

end_datestring nullable

YYYY-MM-DD UTC, inclusive (defaults to today)

api_keystring nullable

Filter to one virtual key token hash

user_idstring nullable

Filter to one canonical internal user recorded on each turn

Response

Successful Response

start_datestring required

Window start day, YYYY-MM-DD UTC, inclusive

end_datestring required

Window end day, YYYY-MM-DD UTC, inclusive

routers_in_scopeinteger required

How many groups this response carries. Every auto-router configured on the proxy counts, whether or not it served anything in the window. To count only the routers that did serve traffic, filter groups to the entries whose sessions is above zero

Changes