Retrieval

Submit a bounded resumable web crawl

Accepts an owner-scoped durable crawl job. max_credits is required; the accepted request and active reader/source revisions are frozen for idempotent replay. A repeated matching idempotency key returns the original job without preparing another reservation.

post/web/crawl

Request

  • Base URL: https://api.crawlora.net/api/v1
  • URL: https://api.crawlora.net/api/v1/web/crawl
  • Auth: API key in header x-api-key

Request body

idempotency_keystring required
sitemap'include' | 'only' | 'skip'
urlstring uri required

Response

Durable crawl job accepted

codeinteger
msgstring

Example response

{
  "code": 202,
  "msg": "Accepted"
}

Changes

No recorded changes to this operation across all 3 revisions of this API.