auto router

Get Auto Router Benchmarks

Benchmarks for the auto-router dashboard: session shape, savings against the configured baseline, and prompt-caching behaviour bucketed by what the router did.

Reads the LiteLLM_AutoRouterSession rollup, folded once per request at spend-write time, so this endpoint never scans LiteLLM_SpendLogs. A session is in the window when it overlaps it: its last turn is on or after start_date and its first turn is on or before end_date. Overall hit rate is over telemetry-bearing turns; each bucket's hit rate is over that bucket's turns.

The rollup supplies the measures, never the list. Which routers appear comes from the model registry, so one shows up as soon as it is configured and reads zero until it serves traffic, and routers_in_scope counts those too rather than only the routers the window recorded.

get/auto_router/benchmarks

Query parameters

start_datestring nullable

YYYY-MM-DD UTC, inclusive (defaults to 30 days before end_date)

YYYY-MM-DD UTC, inclusive (defaults to 30 days before end_date)

end_datestring nullable

YYYY-MM-DD UTC, inclusive (defaults to today)

YYYY-MM-DD UTC, inclusive (defaults to today)

Response

Successful Response

start_datestring required

Window start day, YYYY-MM-DD UTC, inclusive

end_datestring required

Window end day, YYYY-MM-DD UTC, inclusive

routers_in_scopeinteger required

How many groups this response carries. Every auto-router configured on the proxy counts, whether or not it served anything in the window. To count only the routers that did serve traffic, filter groups to the entries whose sessions is above zero

Changes