v2 · Agent-first crawler API

Crawl

Validate or queue a full-site crawl.

The default is dry_run=true to protect users from accidental high-cost crawls. Set dry_run=false explicitly to enqueue.

post/api/v2/crawl

Request

  • Base URL: https://smartcrawler.io (the document declares no server; this is the origin it was published from)
  • URL: https://smartcrawler.io/api/v2/crawl
  • Auth: none declared

Headers

authorizationstring
X-API-Keystring

Request body

urlstring required

Known site root URL.

limitinteger
dry_runboolean

Default true: estimate and validate without queuing a crawl job.

include_pathsstring[]
exclude_pathsstring[]
max_depthinteger
poll_intervalinteger

Response

Successful Response

unknown required

Changes

No recorded changes to this operation across all 1 revision of this API.