v2
v2-discover

Discover

List a site's URLs with no browser (sitemap + HTTP-only crawl).

post/v2/discover

Request

  • Base URL: https://api.enconvert.com (the document declares no server; this is the origin it was published from)
  • URL: https://api.enconvert.com/v2/discover
  • Auth: none declared

Headers

X-API-Keystring nullable
authorizationstring nullable

Request body

urlstring required
mode'sitemap' | 'crawl' | 'hybrid'
max_urlsinteger
max_depthinteger
include_patternsstring[]

Regex (re.search) allowlist; a URL must match at least one to be returned. Empty = allow all.

exclude_patternsstring[]

Regex (re.search) denylist; a URL matching any is dropped. Applied after include_patterns.

same_domain_onlyboolean
respect_robotsboolean

When true, robots.txt Disallow rules are applied to the RETURNED list (disallowed URLs are dropped). NOTE: crawl4ai 0.8.9 has no native robots gate, so in crawl/hybrid mode pages may still be fetched over HTTP and then discarded — robots is enforced at output-filter time, not fetch time.

render_js'auto' | 'never' | 'always'

Browser-rendered discovery for JS/SPA sites (crawl/hybrid modes only). 'auto' renders only when the HTTP-only crawl returns a JS shell (<=1 URL); 'always' forces a bounded browser crawl; 'never' disables it. Uses the singleton Chromium, so it is capped to a few pages (DISCOVER_JS_MAX_PAGES).

Response

Successful Response

urlstring required
mode'sitemap' | 'crawl' | 'hybrid' required
totalinteger required
urlsstring[]
pages_crawledinteger
truncatedboolean

True when more URLs were discovered than max_urls allowed; the list was capped.

robots_respectedboolean
sourcesobject

Raw URL count contributed by each source before dedup/filter, e.g. {'sitemap': 42, 'crawl': 30}.

warningsstring[]

Changes

No recorded changes to this operation across all 1 revision of this API.