---
title: "Crawler Summary"
method: GET
path: "/gtm-internal/crawler/summary"
tags: ["gtm-internal"]
---

# Crawler Summary

`GET /gtm-internal/crawler/summary`

## Request

- Base URL: `https://api-v1.unusual.ai` (the document declares no server; this is the origin it was published from)
- URL: `https://api-v1.unusual.ai/gtm-internal/crawler/summary`
- Auth: OAuth 2
  - Password
  - Token URL: `/auth/login`

Fleet-wide crawler cost and activity over a `?days=` window, plus a daily series.

Always 200s. An empty window returns a well-formed payload with zeros and
`window.runs_begin_at` — "the crawler has no history yet" is data the page has
to render honestly, not an error, and it is the expected state on the day this
ships.

## Query parameters

- `days` integer

## Response `200`

Successful Response

- CrawlerSummaryOut
  - `window` CrawlerWindowOut, required: The reporting window, echoed back — plus when history actually starts.
    - `days` integer, required
    - `since` string, required
    - `until` string, required
    - `runs_begin_at` string, nullable
  - `totals` CrawlerTotalsOut: Raw counted facts over the window. Every rate below is derived from these.
    - `runs` integer
    - `runs_completed` integer
    - `runs_failed` integer
    - `runs_killed` integer
    - `domains_crawled` integer
    - `pages_discovered` integer
    - `pages_fetched` integer
    - `pages_304` integer
    - `pages_changed` integer
    - `pages_removed` integer
    - `llm_pages_classified` integer
    - `llm_input_tokens` integer
    - `llm_output_tokens` integer
    - `error_count` integer
  - `rates` CrawlerRatesOut: Derived rates, each shipped with its own DENOMINATOR. Two different "error rates" are reported because the two questions are different and a single number would silently answer only one: a fleet can have every run complete while thousands of pages 500, and it can have healthy pages while half its runs are killed at the hour mark.
    - `crawls_per_day` number
    - `not_modified_rate` number, nullable
    - `not_modified_denominator` integer
    - `page_error_rate` number, nullable
    - `page_error_denominator` integer
    - `run_failure_rate` number, nullable
    - `run_failure_denominator` integer
  - `cost` CrawlerCostOut: Dollars, and the numbers they came from.
    - `total_usd` number
    - `per_week_usd` number
    - `input_tokens` integer
    - `output_tokens` integer
    - `window_days` integer
  - `cost_model` CrawlerCostModelOut, required: How dollars were derived from tokens — shipped WITH every dollar figure.
    - `model` string, required
    - `input_usd_per_mtok` number, required
    - `output_usd_per_mtok` number, required
    - `formula` string
    - `source` string, required
    - `scope` string
  - `daily` CrawlerDailyPointOut[]
    - `date` string, required
    - `runs` integer
    - `pages_fetched` integer
    - `pages_304` integer
    - `pages_changed` integer
    - `error_count` integer
    - `llm_input_tokens` integer
    - `llm_output_tokens` integer
    - `cost_usd` number
  - `runs_by_trigger` object

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/unusual/apis/dashboard-server.md) - [All operations](https://skmtc.dev/unusual/apis/dashboard-server/llms.txt) - [OpenAPI document](https://skmtc.dev/unusual/apis/dashboard-server/revisions/89f716930ead?raw)