---
title: "The dataset catalogue: counts per lane and day, the latest day's breakdown, the field dictionary and the technology list. Public"
method: GET
path: "/api/datasets"
---

# The dataset catalogue: counts per lane and day, the latest day's breakdown, the field dictionary and the technology list. Public

`GET /api/datasets`

## Request

- Base URL: `https://crawlcheck.io`
- URL: `https://crawlcheck.io/api/datasets`
- Auth: none required

## Response `200`

crawlcheck-datasets

- object, nullable
  - `ok` boolean, nullable
  - `kind` string, nullable
  - `v` number, nullable
  - `updated` string, nullable
  - `timezone` string, nullable
  - `totals` object, nullable
    - `rows` number, nullable
    - `measured` number, nullable
  - `lanes` object[], nullable
    - `id` string, nullable
    - `label` string, nullable
    - `rows` number, nullable
    - `measured` number, nullable
    - `days` number, nullable
  - `days` object[], nullable
    - `day` string, nullable
    - `lanes` object, nullable
      - `saas` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `bytes` number, nullable
      - `gov` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `bytes` number, nullable
      - `local` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `bytes` number, nullable
      - `filler` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `bytes` number, nullable
  - `latest` object, nullable
    - `day` string, nullable
    - `lanes` object, nullable
      - `saas` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `kind` object, nullable
          - `other` number, nullable
          - `ecommerce` number, nullable
          - `saas` number, nullable
          - `publisher` number, nullable
          - `travel` number, nullable
          - `real-estate` number, nullable
          - `docs` number, nullable
          - `local-service` number, nullable
          - `food` number, nullable
          - `gov-edu` number, nullable
          - `finance` number, nullable
          - `nonprofit` number, nullable
          - `legal` number, nullable
        - `state` object, nullable
        - `platform` object, nullable
          - `unknown` number, nullable
          - `wordpress` number, nullable
          - `nextjs` number, nullable
          - `webflow` number, nullable
          - `drupal` number, nullable
          - `squarespace` number, nullable
          - `shopify` number, nullable
        - `grade` object, nullable
        - `llms_txt` number, nullable
        - `ai_blocking` number, nullable
        - `top_tech` array[], nullable
          - unknown[], nullable
            - unknown
      - `gov` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `kind` object, nullable
          - `gov-edu` number, nullable
        - `state` object, nullable
        - `platform` object, nullable
          - `webflow` number, nullable
          - `unknown` number, nullable
          - `wordpress` number, nullable
          - `drupal` number, nullable
          - `wix` number, nullable
          - `squarespace` number, nullable
        - `grade` object, nullable
        - `llms_txt` number, nullable
        - `ai_blocking` number, nullable
        - `top_tech` array[], nullable
          - unknown[], nullable
            - unknown
      - `local` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `kind` object, nullable
          - `local-service` number, nullable
          - `food` number, nullable
          - `publisher` number, nullable
          - `ecommerce` number, nullable
          - `saas` number, nullable
          - `gov-edu` number, nullable
          - `health` number, nullable
          - `travel` number, nullable
        - `state` object, nullable
          - `colorado` number, nullable
        - `platform` object, nullable
          - `squarespace` number, nullable
          - `unknown` number, nullable
          - `wordpress` number, nullable
          - `wix` number, nullable
          - `webflow` number, nullable
          - `shopify` number, nullable
          - `nextjs` number, nullable
          - `drupal` number, nullable
        - `grade` object, nullable
        - `llms_txt` number, nullable
        - `ai_blocking` number, nullable
        - `top_tech` array[], nullable
          - unknown[], nullable
            - unknown
      - `filler` object, nullable
        - `rows` number, nullable
        - `measured` number, nullable
        - `kind` object, nullable
          - `publisher` number, nullable
          - `saas` number, nullable
          - `other` number, nullable
          - `gov-edu` number, nullable
          - `finance` number, nullable
          - `local-service` number, nullable
          - `health` number, nullable
          - `travel` number, nullable
          - `ecommerce` number, nullable
          - `automotive` number, nullable
          - `nonprofit` number, nullable
        - `state` object, nullable
        - `platform` object, nullable
          - `nextjs` number, nullable
          - `unknown` number, nullable
          - `wordpress` number, nullable
          - `drupal` number, nullable
          - `squarespace` number, nullable
          - `wix` number, nullable
          - `webflow` number, nullable
        - `grade` object, nullable
        - `llms_txt` number, nullable
        - `ai_blocking` number, nullable
        - `top_tech` array[], nullable
          - unknown[], nullable
            - unknown
  - `fields` object[], nullable
    - `name` unknown
    - `meaning` string, nullable
  - `tech_detected` string[], nullable
  - `sample` object, nullable
    - `csv` string, nullable
    - `ndjson` string, nullable
    - `rows` number, nullable
  - `files` object, nullable
    - `url` string, nullable
    - `filters` string[], nullable
    - `access` string, nullable
  - `access` object, nullable
    - `page` string, nullable
    - `request` string, nullable

---

[API](https://skmtc.dev/crawlcheck/apis/crawlcheck-api.md) · [All operations](https://skmtc.dev/crawlcheck/apis/crawlcheck-api/llms.txt) · [OpenAPI document](https://skmtc.dev/crawlcheck/apis/crawlcheck-api/revisions/57acbd6d4489?raw)