Compare robots.txt policy with synthetic AI search-crawler responses

Sends bounded public requests using three documented search-crawler user-agent tokens. The requests do not originate from provider IP ranges, authenticate a provider bot, inspect a Cloudflare account, identify the cause of a response, or prove crawling, indexing, citation, ranking, referral traffic, or revenue.

post/api/cloudflare-ai-crawler-check

Request

  • Base URL: https://actablesite.com
  • URL: https://actablesite.com/api/cloudflare-ai-crawler-check
  • Auth: none declared

Request body

urlstring uri required

Public HTTP or HTTPS website URL. Private and local-network targets are rejected.

Response

Robots policy and synthetic-response comparison completed.

urlstring uri required
checkedAtstring date-time required
robotsUrlstring uri required
robotsStatusinteger required
robotsState'found' | 'missing' | 'unreadable' required
cloudflareObservedboolean required
limitsstring required

Changes

No recorded changes to this operation across all 1 revision of this API.