measurement

AI-crawler activity per path

Which paths each crawler fetched over the window and what it got, one row per bot and path, most hits first. Verified hits only. Use it to find where a bot's status4xx / status5xx in getCrawlers came from: status=error.

contentPage is false for files and framework data (_payload.json, scripts, images, feeds) and for sign-in and account pages. An error there is rarely a lost citation: a framework's data file can 404 to everyone by design. contentOnly=1 keeps the pages a person reads.

Per-path detail is kept 90 days; a range reaching further back is clamped, and historyFrom says so.

get/brands/{brandId}/crawlers/paths

Request

  • Base URL: https://shruwd.io/api/v1
  • URL: https://shruwd.io/api/v1/brands/{brandId}/crawlers/paths
  • Auth: HTTP bearer

Query parameters

fromstring date

First day, inclusive. Defaults to 29 days before to. Clamped to the plan's history window.

tostring date

Last day, inclusive. Defaults to today (UTC).

botstring

Only this bot, by its botId from getCrawlers (oai_searchbot, not OAI-SearchBot). An unknown one is 400 unknown_bot.

status'2xx' | '3xx' | '4xx' | '5xx' | 'error'

Only paths that got this answer at least once, ordered by how often. error is 4xx or 5xx.

contentOnly'1'

Only paths with contentPage true.

limitinteger

Response

Paths, most hits in the filtered status first.

historyFromstring date nullable required

Set when the range was clamped: to the plan's history, or to the 90 days per-path detail is kept.

totalPathsinteger required

Rows matching the filters before limit. More than paths.length means the list was cut.

Example response

{
  "paths": [
    {
      "path": "/pricing",
      "botId": "oai_searchbot"
    }
  ]
}

Changes

No recorded changes to this operation across all 1 revision of this API.