SEO

Robots.txt & AI Crawler Checker

See every robots.txt rule that blocks search or AI crawlers, the rule that matches a given path, and the exact line to change. Audit a site's robots.txt, test whether a crawler may fetch a path, and see which search and AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more) are allowed. Costs 10 credits per run. Docs: https://seogeoaeo.ai/tools/robots-checker

post/tools/robots-checker/runs

Request

  • Base URL: https://seogeoaeo.ai/api/v1
  • URL: https://seogeoaeo.ai/api/v1/tools/robots-checker/runs
  • Auth: HTTP bearer

Headers

Idempotency-Keystring

Reuse on retries so the same run is returned instead of charged again. A key only matches a run of the same tool and input, and only while that run is among the workspace's latest 1,000.

Alternative to the idempotencyKey body field.

Request body

idempotencyKeystring

Reuse on retries so the same run is returned instead of charged again. A key only matches a run of the same tool and input, and only while that run is among the workspace's latest 1,000.

maxCreditsinteger

Refuse the run with max_credits_exceeded if it costs more than this.

Example request

{
  "input": {
    "url": "https://example.com"
  }
}

Response

Finished run.

Changes

No recorded changes to this operation across all 1 revision of this API.