Retrieval

Map a public site

Maps a public site into a bounded, approximate URL list from its sitemaps and the seed page's links. The first request needs url; options are scope, sitemap (include, only, skip), max_urls (1 to 5000, default 1000) and page_size (1 to 500, default 100). Continue with cursor, optionally with page_size; the first page's snapshot lasts 30 minutes.

post/web/map

Request

  • Base URL: https://api.crawlora.net/api/v1
  • URL: https://api.crawlora.net/api/v1/web/map
  • Auth: API key in header x-api-key

Request body

cursorstring
max_urlsinteger
page_sizeinteger
sitemapstring
urlstring

Response

OK

codeinteger

Code is the HTTP status code or a custom code used to indicate the result of the request @example 200

msgunknown

Changes

No recorded changes to this operation across all 2 revisions of this API.