Scrape

Start an asynchronous same-site crawl

Changed on

Jobs are stored in Redis and expire after 24 hours.

post/v1/crawl

Request

Request body

urlstring uri required
limitinteger
maxDepthinteger
includePathsstring[]
excludePathsstring[]

Response

Crawl started

successboolean
idstring
urlstring

Status URL

Changes