Sign in
crawl-settings

Update web crawl setting

Updates the setting (name, schedule) for web crawl data sources.

Required scope: `ingestion:manage`

put/crawl-settings/{id}

Path parameters

idstring required

Ingestion setting ID for the web crawl data source

Request body

namestring required

Name of the web crawl data source

rrulestring

Recurrence rule (RFC 5545 RRULE). Empty string or omission removes the existing schedule.

Response

Successfully updated web crawl setting

idstring required

Web crawl ingestion setting ID

namestring required

Name of the web crawl ingestion setting

start_urlstring required

Start URL of the web crawl

path_filtersstring[]

Path filters for crawling

content_patternstring[]

Content patterns for filtering

max_depthinteger required

Maximum crawl depth

max_num_filesinteger nullable

Maximum number of files to crawl

html_onlyboolean required

When true, only HTML files will be downloaded

use_browserboolean required

Whether to use a headless browser for crawling

no_canonical_checkboolean

Whether to disable duplicate filtering by canonical URL

file_extensionsstring[]

File extensions to include (e.g. ".pdf", ".docx")

rrulestring

Recurrence rule (RFC 5545 RRULE)

Changes

Changed in 1 of the 71 revisions of this API.1

Of the 71 revisions, 18 have no diff computed.