Domain

Agent-readiness probe

Whether a site publishes the agent-facing files AI agents look for: /llms.txt, /agents.md, and the /.well-known/ucp Universal Commerce Protocol merchant profile (with its versions, capabilities, payment handlers, and MCP endpoint when advertised). Follows redirects; HTML soft-404 pages count as absent. Shopify stores publish all three by default.

Credits: 1 (refunded on failure).

get/api/v1/agent-ready

Request

  • Base URL: https://scrapewhale.dev
  • URL: https://scrapewhale.dev/api/v1/agent-ready
  • Auth: one of:
    • HTTP bearer
    • API key in header x-api-key

Query parameters

domainstring required

The site to probe. Accepts a bare domain or a full URL.

format'json' | 'markdown' | 'both'

Response shape: "json" (structured data only), "markdown" (markdown only), or "both".

Response

Success.

idstring required
status'completed' required
extractorstring required

Extractor name, e.g. "traffic", "meta-ads", "instagram-profile".

titlestring nullable
sourceUrlstring nullable

Canonical public URL of the extracted source.

creditsChargedinteger required

0 when served from the shared cache.

cachedboolean

True when served from the shared extraction cache (free).

createdAtinteger required

Unix epoch milliseconds.

dataobject nullable

Raw structured payload from the source.

markdownstring

Clean markdown rendering of the payload.

Changes

No recorded changes to this operation across all 1 revision of this API.