Crawler

Get config

Get a crawling configuration by ID

get/v4/crawl/config/{configId}

Request

  • Base URL: https://api.kadoa.com
  • URL: https://api.kadoa.com/v4/crawl/config/{configId}
  • Auth: API key in header x-api-key

Path parameters

configIdstring required

Response

200

urlstring uri

Single URL to start crawling (for backward compatibility)

startUrlsstring[]

List of URLs for crawling

pathsFilterInstring[]

Regex patterns to include specific full URLs. Accepts array or JSON-stringified array of regex source strings.

pathsFilterOutstring[]

Regex patterns to exclude specific full URLs. Accepts array or JSON-stringified array of regex source strings.

proxyTypestring nullable

Type of proxy to use

proxyCountrystring nullable

Country for proxy selection

timeoutnumber

Timeout in milliseconds

maxDepthnumber

Maximum crawling depth

maxPagesnumber

Maximum number of pages to crawl

maxMatchesnumber

Maximum number of matched pages to crawl before stopping

concurrencynumber

Number of concurrent crawlers

strictDomainboolean

Whether to stay within the same domain

loadImagesboolean

Whether to load images during crawling

safeModeboolean

Enable safe mode for crawling

callbackUrlstring uri nullable

Webhook URL for completion notifications

processDuringCrawlboolean

Whether to run preprocessing and extraction during the crawl phase

matchThresholdnumber

Match threshold override for blueprint filtering

rawModeboolean

Whether this is a raw data mode crawl

jobIdstring

Internal: Job ID for workflow tracking

workflowIdstring

Internal: Owning workflow (scraperId) for BrightData cost attribution

dataKeystring

Internal: Data key for Parquet storage path

billingSourcestring

Internal: Billing source identifier

configIdstring required

Config ID

userIdstring required

User ID

createdAtstring required

Created at

Changes

No recorded changes to this operation across all 1 revision of this API.