Start Crawl

Crawl a site once, without making a project for it.

The crawl is a project underneath -- hidden, kept for a day, with the same sitemap reading, ladder, scope rules and formats a project run has -- so POST /crawl/{id}/keep turns this one job into a watched project with its pages and its first run already in place. Answers 202 with the id to poll at GET /crawl/{id}; webhook is told instead.

post/api/v1/crawl

Request

  • Base URL: https://api.mesharc.dev (the document declares no server; this is the origin it was published from)
  • URL: https://api.mesharc.dev/api/v1/crawl
  • Auth: none declared

Headers

authorizationstring nullable
x-mesharc-tenantstring nullable

Request body

urlstring required
limitinteger nullable
maxDepthinteger nullable
includePathsstring[] nullable
excludePathsstring[] nullable
allowSubdomainsboolean nullable
useSitemapboolean nullable
sitemapModestring nullable
crawlModestring nullable
maxAgeinteger nullable
maxTierstring nullable
delayinteger nullable
concurrencyinteger nullable
respectRobotsboolean nullable
scrapeOptionsobject nullable
configobject nullable
destinationstring nullable

Response

Successful Response

unknown required

Changes

No recorded changes to this operation across all 1 revision of this API.