---
title: "Check robots.txt rules for AI crawlers"
method: POST
path: "/websites/checkAiRobots"
tags: ["Websites"]
---

# Check robots.txt rules for AI crawlers

`POST /websites/checkAiRobots`

## Request

- Base URL: `https://api.databuddy.cc`
- URL: `https://api.databuddy.cc/websites/checkAiRobots`
- Auth: API key in header `x-api-key`

Reads the website's robots.txt and reports, for each AI crawler user agent, whether it is allowed, partly blocked, or blocked, and since when a blocked crawler has been blocked. Requires website read permission.

## Request body

- object
  - `websiteId` string, required
  - `userAgents` string[], required

## Response `200`

OK

- object
  - `hasRobotsTxt` boolean, required
  - `access` string[], required
  - `blockedSince` string[], required

## Other responses

- `400`: 400
- `401`: 401
- `402`: 402
- `403`: 403
- `404`: 404
- `409`: 409
- `429`: 429
- `500`: 500
- `503`: 503

---

[API](https://skmtc.dev/databuddy/apis/databuddy-api.md) - [All operations](https://skmtc.dev/databuddy/apis/databuddy-api/llms.txt) - [OpenAPI document](https://skmtc.dev/databuddy/apis/databuddy-api/revisions/ddfbba648d52?raw)