Sitemap

Submit a sitemap crawl and get a quote

Submit an asynchronous crawl that discovers the URLs on one hostname from a starting URL, and get a cost quote. Nothing is crawled or charged until you call approveSitemapJob, within one hour. Use it to map a site's pages; use fetchUrl to read a page. Also served under /map with the same shapes.

post/sitemap

Request

  • Base URL: https://request.usestring.ai/v1
  • URL: https://request.usestring.ai/v1/sitemap
  • Auth: HTTP bearer

Request body

urlstring uri required

Starting URL. The crawl stays on this hostname.

maxPagesinteger

Maximum pages to crawl.

maxDepthinteger

Maximum link depth from the starting URL.

pathPrefixstring

Only crawl URLs whose path starts with this prefix.

budgetUsdnumber

Spend cap in USD. Defaults to the quote itself, so every job is capped; the crawl stops with token_cap_exceeded before exceeding it.

useSitemapboolean

Also seed the crawl from the site's root /sitemap.xml (one extra quoted and billed page).

Response

Quote produced; job awaiting approval.

jobIdstring required
status'awaiting_approval' required
estimatedCostUsdstring required

Quoted cost in USD, as a decimal string.

estimatedPagesinteger required

Changes

No recorded changes to this operation across all 1 revision of this API.