H2h2o
Document Ingestion

Creates a job to add files from the AWS S3 storage into a collection.

Creates a job to add files from the AWS S3 storage into a collection.

post/ingest/s3/job

Request

Query parameters

collection_idstring required

String id of the collection to add the ingested documents into.

gen_doc_summariesboolean

Whether to auto-generate document summaries (uses LLM).

gen_doc_questionsboolean

Whether to auto-generate sample questions for each document (uses LLM).

audio_input_languagestring

Language of audio files.

ocr_modelstring

Which method to use to extract text from images using AI-enabled optical character recognition (OCR) models. docTR is best for Latin text, PaddleOCR is best for certain non-Latin languages, Tesseract covers a wide range of languages. Mississippi works well on handwriting. - auto - Automatic will auto-select the best OCR model for every page. - off - Disable OCR for speed, but all images will then be skipped (also no image captions will be made).

tesseract_langstring

Which language to use when using ocr_model="tesseract".

keep_tables_as_one_chunkboolean

When tables are identified by the table parser the table tokens will be kept in a single chunk.

chunk_by_pageboolean

Each page will be a chunk. keep_tables_as_one_chunk will be ignored if this is true.

handwriting_checkboolean

Check pages for handwriting. Will use specialized models if handwriting is found.

ingest_mode'standard' | 'agent_only'

Ingest mode to use. - standard - Files will be ingested for use with RAG - agent_only - Bypasses standard ingestion. Files can only be used with agents.

timeoutnumber double

Timeout in seconds

Request body

urlsstring[] required

The path or list of paths of S3 files or directories.

regionstring

The name of the region used for interaction with AWS services.

metadataobject

Metadata for the documents.

Response

Successful operation

idstring required
namestring required
overall_status'in progress' | 'completed' | 'failed' | 'canceled' required
statusstring
passed_percentagenumber double required
failed_percentagenumber double required
progressnumber double required
created_atstring date-time required
updated_atstring date-time required
kindstring required
errorsstring[] required
durationstring required
duration_secondsnumber double required
canceled_bystring
cancel_reasonstring
timeoutnumber double
start_timenumber double
files_totalinteger
files_failedinteger

Changes

No recorded changes to this operation across all 1 revision of this API.