crawl-settings

Update web crawl setting

Changed on

Updates the setting (name, schedule) for web crawl data sources.

Required scope: `ingestion:manage`

put/crawl-settings/{id}

Request

  • Base URL
    https://developer.qaip.com/api/v1
  • URL
    https://developer.qaip.com/api/v1/crawl-settings/{id}
  • Auth
    API key in header x-api-key

Path parameters

idstring required

Ingestion setting ID for the web crawl data source

Request body

namestring required

Name of the web crawl data source

rrulestring

Recurrence rule (RFC 5545 RRULE). Empty string or omission removes the existing schedule.

Response

Successfully updated web crawl setting

idstring required

Web crawl ingestion setting ID

namestring required

Name of the web crawl ingestion setting

start_urlstring required

Start URL of the web crawl

path_filtersstring[]

Path filters for crawling

content_patternstring[]

Content patterns for filtering

max_depthinteger required

Maximum crawl depth

max_num_filesinteger nullable

Maximum number of files to crawl

html_onlyboolean required

When true, only HTML files will be downloaded

use_browserboolean required

Whether to use a headless browser for crawling

no_canonical_checkboolean

Whether to disable duplicate filtering by canonical URL

file_extensionsstring[]

File extensions to include (e.g. ".pdf", ".docx")

rrulestring

Recurrence rule (RFC 5545 RRULE)

Changes

  • 2489c6f1a9d8Revision changes
    • info

      added the non-success response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      added the optional property / to the response with the status

  • b065e544fec5Revision changes
    • info

      added the optional property to the response with the status