ORorthogonal
Tavily

Tavily Crawl is a graph-based website traversal tool that can explore hundreds of paths in parallel with built-in extraction and intelligent discovery.

Changed on
post/tavily/crawl

Request

Request body

urlstring required

The root URL to begin the crawl.

instructionsstring

Natural language instructions for the crawler. When specified, the mapping cost increases to 2 API credits per 10 successful pages instead of 1 API credit per 10 pages.

chunks_per_sourceinteger

Chunks are short content snippets (maximum 500 characters each) pulled directly from the source. Use chunks_per_source to define the maximum number of relevant chunks returned per source and to control the raw_content length. Chunks will appear in the raw_content field as: <chunk 1> [...] <chunk 2> [...] <chunk 3>. Available only when instructions are provided. Must be between 1 and 5.

max_depthinteger

Max depth of the crawl. Defines how far from the base URL the crawler can explore.

max_breadthinteger

Max number of links to follow per level of the tree (i.e., per page).

limitinteger

Total number of links the crawler will process before stopping.

select_pathsstring[]

Regex patterns to select only URLs with specific path patterns (e.g., /docs/., /api/v1.).

select_domainsstring[]

Regex patterns to select crawling to specific domains or subdomains (e.g., ^docs.example.com$).

exclude_pathsstring[]

Regex patterns to exclude URLs with specific path patterns (e.g., /private/., /admin/.).

exclude_domainsstring[]

Regex patterns to exclude specific domains or subdomains from crawling (e.g., ^private.example.com$).

allow_externalboolean

Whether to include external domain links in the final results list.

include_imagesboolean

Whether to include images in the crawl results.

extract_depthstring

Advanced extraction retrieves more data, including tables and embedded content, with higher success but may increase latency. basic extraction costs 1 credit per 5 successful extractions, while advanced extraction costs 2 credits per 5 successful extractions.

formatstring

The format of the extracted web page content. markdown returns content in markdown format. text returns plain text and may increase latency.

include_faviconboolean

Whether to include the favicon URL for each result.

timeoutstring

Maximum time in seconds to wait for the crawl operation before timing out. Must be between 10 and 150 seconds.

Response

Successful response

Changes