Crawls and ingest a URL into a collection.

String id of the collection to add the ingested documents into.

Whether to import all web pages linked from this URL will be imported. External links will be ignored. Links to other pages on the same domain will be followed as long as they are at the same level or below the URL you specify. Each page will be transformed into a PDF document.

Max depth of recursion when following links, only when follow_links is true. Max_depth of 0 means don't follow any links, max_depth of 1 means follow only top-level links, etc. Use -1 for automatic (system settings).

Max number of documents when following links, only when follow_links is true. Use None for automatic (system defaults). Use -1 for max (system limit).

Whether to auto-generate document summaries (uses LLM).

Whether to auto-generate sample questions for each document (uses LLM).

Language of audio files.

Which method to use to extract text from images using AI-enabled optical character recognition (OCR) models. docTR is best for Latin text, PaddleOCR is best for certain non-Latin languages, Tesseract covers a wide range of languages. Mississippi works well on handwriting. - auto - Automatic will auto-select the best OCR model for every page. - off - Disable OCR for speed, but all images will then be skipped (also no image captions will be made).

Which language to use when using ocr_model="tesseract".

When tables are identified by the table parser the table tokens will be kept in a single chunk.

Each page will be a chunk. keep_tables_as_one_chunk will be ignored if this is true.

Check pages for handwriting. Will use specialized models if handwriting is found.

Ingest mode to use. - standard - Files will be ingested for use with RAG - agent_only - Bypasses standard ingestion. Files can only be used with agents.

Timeout in seconds