MEmercury-hq
robots-txt
ai-crawler-policy
crawler-permission-audit
llms-txt
ai-txt
tdm-opt-out
eu-ai-act-compliance
ai-bot-blocking
training-data-consent
verifiable-policy-snapshot
provenance
attestation
keyless
agent-native
data-feed

AI Crawler Permission Audit — signed robots.txt / llms.txt / ai.txt verdict (keyless x402)

Domain → signed, timestamped per-AI-crawler allow/block audit from robots.txt + llms.txt + ai.txt (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, …). Deterministic, no LLM. EU-AI-Act / TDM opt-out evidence: the signed verdict proves the crawler policy as it stood at fetch time.

get/buy/robots

Request

Query parameters

domainstring required

domain or URL to audit (e.g. example.com or https://example.com/page); only the origin is used

pathstring

optional path to evaluate the verdict for (default '/', the whole-site question)

Response

Delivered after the x402 payment settles on Base mainnet.

okboolean required

true on success; false on an honest failure (still delivered, never charged for a stub)

urlstring required

origin audited (https://host)

statusinteger

HTTP status of the robots.txt fetch

textstring

canonical JSON the signed attestation is computed over (data, keys sorted)

fetchedAtstring

ISO-8601 fetch time — the 'as-of' moment the signed policy snapshot proves

errorstring

present only when ok:false

Changes

No recorded changes to this operation across all 1 revision of this API.