analyze

Robots Check Ep

robots.txt as a crawler reads it: raw file, parsed groups, Sitemap lines, and an allowed/blocked matrix for search and AI bots (Googlebot, Bingbot, GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, Claude-SearchBot, Google-Extended, CCBot …) over the homepage plus any URLs given. Also reports /llms.txt. Any public domain, no crawl.

post/analyze/robots

Request

  • Base URL: https://api.xeorank.com (the document declares no server; this is the origin it was published from)
  • URL: https://api.xeorank.com/analyze/robots
  • Auth: API key in header X-Agent-Key

Query parameters

emailstring nullable

Request body

domainstring required
urlsstring[] nullable
botsstring[] nullable
max_urlsinteger nullable
status_sampleinteger nullable
pathstring nullable

Response

Successful Response

object required

Changes

No recorded changes to this operation across all 1 revision of this API.