Web

Crawl a site

Crawls a site to markdown across multiple pages. Asynchronous — returns a queued job; poll GET /v1/jobs/{id} for the pages. Charges credits on completion.

post/web/crawl

Request

  • Base URL: https://api.crevio.co/v1
  • URL: https://api.crevio.co/v1/web/crawl
  • Auth: API key in header Authorization

Headers

Idempotency-Keystring

Makes this write safe to retry: a repeat with the same key replays the first response instead of creating a second resource. Scoped to the account, remembered for 24 hours; reusing a key with a different payload is an error.

Request body

urlstring required

Starting URL.

max_pagesinteger

Maximum pages to crawl (default 10, capped at 100).

max_depthinteger

Maximum link depth (default 2).

Response

Job queued

idstring required
objectstring required
typestring required
status'queued' | 'processing' | 'completed' | 'failed' | 'cancelled' required
modelstring required
progressinteger required
inputobject required
resultobject nullable required
errorstring nullable required
error_codestring nullable required
credits_usedinteger nullable required
provider_request_idstring nullable required
created_atstring date-time required
started_atstring date-time nullable required
completed_atstring date-time nullable required

Changes

No recorded changes to this operation across all 2 revisions of this API.