Crawler

Start crawl

Create a crawling configuration and start a session in one operation (equivalent to v4/crawl)

post/v4/crawl/

Request

Base URL: https://api.kadoa.com
URL: https://api.kadoa.com/v4/crawl/
Auth: API key in header x-api-key

Request body

urlstring uri

Single URL to start crawling (for backward compatibility)

startUrlsstring[]

List of URLs for crawling

pathsFilterInstring[]

Regex patterns to include specific full URLs. Accepts array or JSON-stringified array of regex source strings.

pathsFilterOutstring[]

Regex patterns to exclude specific full URLs. Accepts array or JSON-stringified array of regex source strings.

proxyTypestring nullable

Type of proxy to use

proxyCountrystring nullable

Country for proxy selection

timeoutnumber

Timeout in milliseconds

maxDepthnumber

Maximum crawling depth

maxPagesnumber

Maximum number of pages to crawl

maxMatchesnumber

Maximum number of matched pages to crawl before stopping

concurrencynumber

Number of concurrent crawlers

strictDomainboolean

Whether to stay within the same domain

loadImagesboolean

Whether to load images during crawling

safeModeboolean

Enable safe mode for crawling

callbackUrlstring uri nullable

Webhook URL for completion notifications

processDuringCrawlboolean

Whether to run preprocessing and extraction during the crawl phase

matchThresholdnumber

Match threshold override for blueprint filtering

rawModeboolean

Whether this is a raw data mode crawl

jobIdstring

Internal: Job ID for workflow tracking

workflowIdstring

Internal: Owning workflow (scraperId) for BrightData cost attribution

dataKeystring

Internal: Data key for Parquet storage path

billingSourcestring

Internal: Billing source identifier

Response

200

messagestring required

Response message

sessionIdstring required

Session ID

configIdstring

Config ID (included when creating config and starting session)

errorstring nullable required

Error message if any

Changes

No recorded changes to this operation across all 1 revision of this API.