---
title: "The agentic web map (?month=YYYY-MM; default this month, month to date until sealed): every reachable site of a census frame (top 10,000, else top 1,000) placed in four layers - DNS host (NS records), network (A record, origin autonomous system holder), bot protection and site platform (technology record) - with what the census saw it do to AI crawlers: refused an AI identity at the edge while serving the browser, blocked one in robots.txt, served llms.txt, declared Content-Signal. Per provider (named at 10+ sites), links between layers split by those outcomes, and where refusals and blocks concentrate. ?domain= returns one domain's four layers instead (signed crawlcheck-agentic-web-map-lookup; live read if the domain is not in the map, 60 an hour per address). No site named in the edition. Signed crawlcheck-agentic-web-map; sealed editions at /api/v1/agentic-web-map/YYYY-MM.json."
method: GET
path: "/api/v1/agentic-web-map"
---

# The agentic web map (?month=YYYY-MM; default this month, month to date until sealed): every reachable site of a census frame (top 10,000, else top 1,000) placed in four layers - DNS host (NS records), network (A record, origin autonomous system holder), bot protection and site platform (technology record) - with what the census saw it do to AI crawlers: refused an AI identity at the edge while serving the browser, blocked one in robots.txt, served llms.txt, declared Content-Signal. Per provider (named at 10+ sites), links between layers split by those outcomes, and where refusals and blocks concentrate. ?domain= returns one domain's four layers instead (signed crawlcheck-agentic-web-map-lookup; live read if the domain is not in the map, 60 an hour per address). No site named in the edition. Signed crawlcheck-agentic-web-map; sealed editions at /api/v1/agentic-web-map/YYYY-MM.json.

`GET /api/v1/agentic-web-map`

## Request

- Base URL: `https://crawlcheck.io`
- URL: `https://crawlcheck.io/api/v1/agentic-web-map`
- Auth: none required

## Query parameters

- `month` string
- `domain` string

## Response `200`

crawlcheck-agentic-web-map or crawlcheck-agentic-web-map-lookup

- object, nullable
  - `kind` string, nullable
  - `v` number, nullable
  - `month` string, nullable
  - `state` string, nullable
  - `frame` string, nullable
  - `run` number, nullable
  - `sites` number, nullable
  - `mapped` number, nullable
  - `totals` object, nullable
    - `sites` number, nullable
    - `edge_refused` object, nullable
      - `any` object, nullable
        - `count` number, nullable
        - `pct` number, nullable
    - `robots` object, nullable
      - `served` number, nullable
      - `block_any` object, nullable
        - `count` number, nullable
        - `pct` number, nullable
    - `llms_txt` object, nullable
      - `count` number, nullable
      - `pct` number, nullable
  - `layers` object[], nullable
    - `id` string, nullable
    - `label` string, nullable
    - `providers_named` number, nullable
    - `nodes` object[], nullable
      - `id` string, nullable
      - `name` string, nullable
      - `kind` string, nullable
      - `share` number, nullable
      - `sites` number, nullable
      - `edge_refused` object, nullable
        - `any` object, nullable
          - `count` number, nullable
          - `pct` number, nullable
      - `robots` object, nullable
        - `served` number, nullable
        - `block_any` object, nullable
          - `count` number, nullable
          - `pct` number, nullable
      - `llms_txt` object, nullable
        - `count` number, nullable
        - `pct` number, nullable
  - `links` object[], nullable
    - `from` string, nullable
    - `to` string, nullable
    - `sites` number, nullable
    - `edge_refused` number, nullable
    - `robots_only` number, nullable
    - `passes` number, nullable
  - `attribution` object, nullable
    - `edge_refusals_by_network` object[], nullable
      - `node` string, nullable
      - `name` string, nullable
      - `count` number, nullable
      - `share_of_all` number, nullable
      - `rate` number, nullable
      - `overall_rate` number, nullable
      - `lift` number, nullable
  - `sha256` unknown
  - `signature` object, nullable
    - `v` number, nullable
    - `alg` string, nullable
    - `kid` unknown
    - `sig` unknown

## Other responses

- `400`: bad month or domain
- `404`: no edition for that month yet
- `429`: live lookup limit

---

[API](https://skmtc.dev/crawlcheck/apis/crawlcheck-api.md) · [All operations](https://skmtc.dev/crawlcheck/apis/crawlcheck-api/llms.txt) · [OpenAPI document](https://skmtc.dev/crawlcheck/apis/crawlcheck-api/revisions/57acbd6d4489?raw)