Crawl a website: fetch a page, follow the links inside the same host and return every page it reached, within a page budget you set. For reading documentation, a product catalogue or a changelog in one paid call instead of many.

Paid per call via x402. Send without payment to get the exact price in the 402.

post/v1/tools/site-crawl

Request

  • Base URL: https://agents.dexl.io
  • URL: https://agents.dexl.io/v1/tools/site-crawl
  • Auth: API key in header X-Payment

Request body

urlstring required

Absolute http(s) URL to start from

max_pagesinteger

Response

The tool output

Changes

No recorded changes to this operation across all 1 revision of this API.