---
title: "Get Auto Router Benchmarks"
method: GET
path: "/auto_router/benchmarks"
tags: ["auto router"]
---

# Get Auto Router Benchmarks

`GET /auto_router/benchmarks`

Benchmarks for the auto-router dashboard: session shape, savings against the configured
baseline, and prompt-caching behaviour bucketed by what the router did.

Reads the LiteLLM_AutoRouterSession rollup, folded once per request at spend-write time,
so this endpoint never scans LiteLLM_SpendLogs. A session is in the window when it
overlaps it: its last turn is on or after start_date and its first turn is on or before
end_date. Overall hit rate is over telemetry-bearing turns; each bucket's hit rate is
over that bucket's turns.

The rollup supplies the measures, never the list. Which routers appear comes from the
model registry, so one shows up as soon as it is configured and reads zero until it
serves traffic, and `routers_in_scope` counts those too rather than only the routers the
window recorded.

## Query parameters

- `start_date` string, nullable — YYYY-MM-DD UTC, inclusive (defaults to 30 days before end_date)
- `end_date` string, nullable — YYYY-MM-DD UTC, inclusive (defaults to today)

## Response `200`

Successful Response

- AutoRouterBenchmarksResponse — Benchmarks for the auto-router dashboard, aggregated from the per-session rollup.
  - `start_date` string, required — Window start day, YYYY-MM-DD UTC, inclusive
  - `end_date` string, required — Window end day, YYYY-MM-DD UTC, inclusive
  - `routers_in_scope` integer, required — How many groups this response carries. Every auto-router configured on the proxy counts, whether or not it served anything in the window. To count only the routers that did serve traffic, filter `groups` to the entries whose `sessions` is above zero
  - `totals` AutoRouterBenchmarkTotals, required — Session-shape and savings aggregates over auto-routed traffic in the window.
    - `sessions` integer, required
    - `turns` integer, required
    - `avg_turns_per_session` number, required
    - `avg_session_seconds` number, required
    - `avg_tokens_per_session` number, required
    - `spend` number, required — What the routed traffic actually cost
    - `saved_spend` number, required — Signed dollars saved versus each router's savings baseline (derived from its hardest tier, or the configured override), from the same per-request savings record the usage tab reads
    - `baseline_spend` number, required — spend plus saved_spend: the estimated single-model cost
    - `saved_pct` number, required — saved_spend over baseline_spend, as a percentage
    - `saved_per_session` number, required
    - `cache` AutoRouterCacheStats, required — Prompt-caching behaviour of auto-routed turns, bucketed by what the router did. Every in-order turn falls in exactly one bucket: the session stayed on the same model, visited a model for the first time (cold by design), or returned to a model it had already used. Out-of-order turns (cross-pod flush races) are counted but not bucketed.
      - `coverage_pct` number, required — Share of turns that carried cache telemetry
      - `hit_rate_pct` number, required — All cache hits over telemetry-bearing turns
      - `same_model` AutoRouterCacheBucket, required — One prompt-caching bucket of turns, with how often those turns hit the cache.
        - `turns` integer, required — Turns classified into this bucket
        - `hits` integer, required — Turns in this bucket whose response reported cache-read tokens
        - `hit_rate_pct` number, required — hits over this bucket's turns, as a percentage
      - `first_visit` AutoRouterCacheBucket, required — One prompt-caching bucket of turns, with how often those turns hit the cache.
        - `turns` integer, required — Turns classified into this bucket
        - `hits` integer, required — Turns in this bucket whose response reported cache-read tokens
        - `hit_rate_pct` number, required — hits over this bucket's turns, as a percentage
      - `return_to_tier` AutoRouterCacheBucket, required — One prompt-caching bucket of turns, with how often those turns hit the cache.
        - `turns` integer, required — Turns classified into this bucket
        - `hits` integer, required — Turns in this bucket whose response reported cache-read tokens
        - `hit_rate_pct` number, required — hits over this bucket's turns, as a percentage
      - `unordered_turns` integer, required — Turns that arrived out of order and were not bucketed
      - `return_misses_expired` integer, required — Return-to-tier misses where the model's recorded cache TTL had lapsed
      - `return_misses_within_ttl` integer, required — Return-to-tier misses inside the recorded TTL: the prefix changed or the provider evicted the entry early; billing telemetry cannot distinguish the two
      - `return_misses_unknown` integer, required — Return-to-tier misses with no recorded TTL to attribute against
      - `ttl_5m_turns` integer, required — Turns whose cache write used the five-minute TTL
      - `ttl_1h_turns` integer, required — Turns whose cache write used the one-hour TTL
  - `groups` AutoRouterBenchmarkGroup[], required — One entry per auto-router, listed from the model registry rather than from the rollup, so a router appears as soon as it is configured and reads zero until it serves traffic. Semantic auto-routers are absent: they record no routing decision, so no session can ever be attributed to them
    - `sessions` integer, required
    - `turns` integer, required
    - `avg_turns_per_session` number, required
    - `avg_session_seconds` number, required
    - `avg_tokens_per_session` number, required
    - `spend` number, required — What the routed traffic actually cost
    - `saved_spend` number, required — Signed dollars saved versus each router's savings baseline (derived from its hardest tier, or the configured override), from the same per-request savings record the usage tab reads
    - `baseline_spend` number, required — spend plus saved_spend: the estimated single-model cost
    - `saved_pct` number, required — saved_spend over baseline_spend, as a percentage
    - `saved_per_session` number, required
    - `cache` AutoRouterCacheStats, required — Prompt-caching behaviour of auto-routed turns, bucketed by what the router did. Every in-order turn falls in exactly one bucket: the session stayed on the same model, visited a model for the first time (cold by design), or returned to a model it had already used. Out-of-order turns (cross-pod flush races) are counted but not bucketed.
      - `coverage_pct` number, required — Share of turns that carried cache telemetry
      - `hit_rate_pct` number, required — All cache hits over telemetry-bearing turns
      - `same_model` AutoRouterCacheBucket, required — One prompt-caching bucket of turns, with how often those turns hit the cache.
        - `turns` integer, required — Turns classified into this bucket
        - `hits` integer, required — Turns in this bucket whose response reported cache-read tokens
        - `hit_rate_pct` number, required — hits over this bucket's turns, as a percentage
      - `first_visit` AutoRouterCacheBucket, required — One prompt-caching bucket of turns, with how often those turns hit the cache.
        - `turns` integer, required — Turns classified into this bucket
        - `hits` integer, required — Turns in this bucket whose response reported cache-read tokens
        - `hit_rate_pct` number, required — hits over this bucket's turns, as a percentage
      - `return_to_tier` AutoRouterCacheBucket, required — One prompt-caching bucket of turns, with how often those turns hit the cache.
        - `turns` integer, required — Turns classified into this bucket
        - `hits` integer, required — Turns in this bucket whose response reported cache-read tokens
        - `hit_rate_pct` number, required — hits over this bucket's turns, as a percentage
      - `unordered_turns` integer, required — Turns that arrived out of order and were not bucketed
      - `return_misses_expired` integer, required — Return-to-tier misses where the model's recorded cache TTL had lapsed
      - `return_misses_within_ttl` integer, required — Return-to-tier misses inside the recorded TTL: the prefix changed or the provider evicted the entry early; billing telemetry cannot distinguish the two
      - `return_misses_unknown` integer, required — Return-to-tier misses with no recorded TTL to attribute against
      - `ttl_5m_turns` integer, required — Turns whose cache write used the five-minute TTL
      - `ttl_1h_turns` integer, required — Turns whose cache write used the one-hour TTL
    - `router_name` string, required — The auto-router alias requests were sent to
    - `router_type` string, required — complexity, adaptive or quality
    - `tier_turns` object — Turns per tier, keyed by the tier name the routing decision recorded at request time (never re-derived at read time, since the tier-to-model mapping is mutable config). Tier names are scoped to this group's router_type and are not comparable across types: a complexity router reports 'SIMPLE'/'MEDIUM'/'COMPLEX'/'REASONING', a quality router reports its numeric quality tier, and an adaptive router records no tier at all. Turns no tier served (the classifier fell back to default_model) are absent rather than pooled under a sentinel key, so the values may sum to less than turns

## Other responses

- `422` — Validation Error

## Changes

- **2026-09-18** `082b5fabd909` — 1 info
  - endpoint added

[Change history](https://skmtc.dev/flock/apis/litellm-api/changes/auto_router/benchmarks/get.md)

---

[API](https://skmtc.dev/flock/apis/litellm-api.md) · [All operations](https://skmtc.dev/flock/apis/litellm-api/llms.txt) · [OpenAPI document](https://skmtc.dev/flock/apis/litellm-api/revisions/3b2b241e4b0b?raw)