Index an explicit list of objects from a Supabase Storage bucket, given as a JSON Lines manifest in the same bucket. One bulk job; nothing is listed or scanned.

One object per line, UTF-8, uncompressed. uri is required, the rest are optional:

{"uri": "s3://documents/inbox/2026-q4-msa.pdf", "source_identity": "contract-8841", "custom_metadata": {"department": "legal", "counterparty": "Northwind"}}
{"uri": "s3://documents/inbox/2026-q4-sow.pdf", "source_identity": "contract-8842", "size": 482113, "etag": "9b2cf535f27731c974343645a3985328"}
{"uri": "s3://documents/archive/2025-renewal.pdf", "source_identity": "contract-7310"}
{"uri": "s3://documents/inbox/handbook.pdf"}

Physical location vs. logical identity

uri is where the file currently lives: this bucket, this key. It must be inside bucket_name, and it is the only field Captain reads bytes from.

source_identity is what the document is, in your own terms. Captain keeps one document per identity, so the identity is what decides whether a later index is an update or a new document:

• Give s3://documents/inbox/2026-q4-msa.pdf the identity contract-8841. Move the file to s3://documents/archive/2026-q4-msa.pdf and index it again with the same identity, and Captain updates that one document. Its chunks are replaced and the old ones are retired.
• Omit source_identity and the identity is the uri. The same file at a new key is a different identity, so you get a second document and both versions answer queries.

A repeat of an identity already in the collection is skipped while skip_existing is true (the default), and replaced when you send overwrite_existing: true or skip_existing: false. While it is skipping, Captain compares content. A repeat whose content is unchanged is skipped. A repeat whose content differs is an identity conflict: it is neither skipped nor indexed, and the job reports it under identity_conflicts so that you can send it again with overwrite_existing: true. When either version has no checksum, the repeat is skipped. Use your own record id, not a path, whenever content can move or be delivered twice.

The other line fields

• custom_metadata: merged over the job-level custom_metadata, key by key, for this file only. A line's value wins on a shared key.
• checksum and version_id: recorded on the document. A checksum that is the object's ETag, written bare or as etag:..., is compared with the object before anything is read, and a line that does not match is reported as unreachable instead of being indexed. Write any other algorithm with its prefix, for example sha256:...; it is recorded but not verified. version_id pins the object version that is read.
• size and etag: when both are present, as in a bucket listing export, Captain does not read the object's metadata before indexing. That is what lets a large manifest start quickly.

Limits and error handling

A manifest holds up to 1,000,000 lines and 500 MB, with each line at most 10 KB. It must be plain UTF-8 .jsonl or .ndjson; a compressed body is refused. Per line, custom_metadata takes at most 50 keys, keys up to 64 characters and string values up to 1,024 characters. The job-level custom_metadata is capped at 64 KB serialized. It is attached to every file in the job, so metadata size times file count must stay under 2 GB: a few hundred bytes fits the full million lines, and the full 64 KB fits about 32,000 files. Larger jobs should carry less job-level metadata or be split.

A wrong manifest_uri is a 400 before the job is created, as is a manifest larger than the size cap. Request bodies are capped at 1 MB, which is why a large object list is a manifest in storage rather than an inline array.

Unusable lines (invalid JSON, not a JSON object, a uri outside the bucket, a duplicate uri, an unknown key) and unreachable objects (missing, access denied, unsupported file type) are counted on the job. GET /v2/jobs/{job_id} returns a manifest object with the totals and a sample of up to 20 of each, giving the line number or uri and a reason. Neither fails the job: a million-line manifest with a few thousand bad lines indexes the rest.

Headers:

• Authorization: Bearer {api_key} - Captain API key for authentication
• X-Organization-ID: Organization UUID

Returns: { job_id, status: "pending" }