---
title: "AI-crawler activity per path"
method: GET
path: "/brands/{brandId}/crawlers/paths"
tags: ["measurement"]
---

# AI-crawler activity per path

`GET /brands/{brandId}/crawlers/paths`

## Request

- Base URL: `https://shruwd.io/api/v1`
- URL: `https://shruwd.io/api/v1/brands/{brandId}/crawlers/paths`
- Auth: HTTP bearer

Which paths each crawler fetched over the window and what it got, one
row per bot and path, most hits first. Verified hits only. Use it to
find where a bot's `status4xx` / `status5xx` in `getCrawlers` came from:
`status=error`.

`contentPage` is false for files and framework data (`_payload.json`,
scripts, images, feeds) and for sign-in and account pages. An error
there is rarely a lost citation: a framework's data file can 404 to
everyone by design. `contentOnly=1` keeps the pages a person reads.

Per-path detail is kept 90 days; a range reaching further back is
clamped, and `historyFrom` says so.

## Query parameters

- `from` string, date
- `to` string, date
- `bot` string
- `status` '2xx' | '3xx' | '4xx' | '5xx' | 'error'
- `contentOnly` '1'
- `limit` integer

## Response `200`

Paths, most hits in the filtered status first.

- CrawlerPaths
  - `brand` object, required
    - `brandId` string, required
    - `name` string, required
  - `range` object, required
    - `from` string, date, required
    - `to` string, date, required
  - `historyFrom` string, date, nullable, required: Set when the range was clamped: to the plan's history, or to the 90 days per-path detail is kept.
  - `totalPaths` integer, required: Rows matching the filters before `limit`. More than `paths.length` means the list was cut.
  - `paths` CrawlerPathActivity[], required
    - `path` string, required
    - `botId` string, required
    - `purpose` 'training' | 'live_retrieval', required
    - `vendor` string, required
    - `contentPage` boolean, required: A page a person reads. False for files and framework data such as `_payload.json`, and for sign-in and account pages.
    - `hits` integer, required: Verified hits by this bot on this path.
    - `status2xx` integer, required
    - `status3xx` integer, required
    - `status4xx` integer, required
    - `status5xx` integer, required
    - `worstStatus` integer, required: The highest status the bot got here in the range. Tells a 404 (no such page) from a 403 (refused).
    - `days` integer, required: Days in the range the bot fetched this path.
    - `lastSeen` string, date, required

## Other responses

- `400`: `invalid_status`, or `unknown_bot` with the known ids in `details.known`.
- `404`: `not_found`: no such resource in this workspace.
- `default`: Every failure has this body.

---

[API](https://skmtc.dev/shruwd/apis/shruwd-api.md) · [All operations](https://skmtc.dev/shruwd/apis/shruwd-api/llms.txt) · [OpenAPI document](https://skmtc.dev/shruwd/apis/shruwd-api/revisions/64a5da079928?raw)