Crawler

Start Crawl

Start a new crawl job.

post/v1/crawl

Request body

urlstring required

depthinteger

breadthinteger nullable

Maximum number of links to crawl per depth level. If None, unlimited (default). Ignored when sitemap=True.

max_pagesinteger nullable

promptstring nullable

schemaobject nullable

sitemapboolean

render_heavy_jsboolean

extraction_modeboolean

stealthboolean

Enable stealth mode to avoid bot detection

mockboolean

webhook_urlstring nullable

Webhook URL to send the job result to

reader_modeboolean

wait_msinteger

country_codestring nullable

The country code to use for fetching (e.g. US, GB)

Successful Response

{"stackTrail":"paths:/v1/crawl:post:responses:200:content:application/json:schema","oasType":"schema","type":"unknown"}

No recorded changes to this endpoint across all 1 revision of this API.