Artifacts
Artifacts

Retrieve Full Document Content in Chunks

Retrieve full document content split into chat-optimized chunks.

This endpoint provides access to complete document content split into manageable chunks suitable for chat interfaces. Unlike semantic chunk retrieval, this returns complete documents divided sequentially.

Key Features:

  • Chat-Optimized Chunking: Documents split into conversational pieces
  • Node Type Filtering: Include or exclude images, tables, and other node types
  • Token-Aware Splitting: Respects token limits for chat context management
  • Sequential Chunks: Maintains document order and narrative flow
  • Filtered Retrieval: Select specific documents using metadata filters
  • Token Limiting: Optional max_tokens parameter to control response size

Content Filtering:

  • Use include to retrieve only specific node types
  • Use exclude to omit unwanted content types
  • Supports TextNode, ImageNode, TableNode, and other node types
  • Filtering is applied before chunking

Chunking Process:

  1. Retrieve filtered documents based on context criteria
  2. Apply node type filters (include/exclude)
  3. Split documents into chat-appropriate segments respecting max_tokens
  4. Return structured chunks with metadata and citations

Notes:

  • Chunks maintain document structure and logical flow
  • Token limiting prevents context window overflow
  • Node type filtering reduces payload size and improves relevance
  • Use /artifacts/search endpoint for semantic search instead
post/v1/artifacts/chunked-content

Request body

format'object' | 'markdown'

Enumeration of content retrieval formats.

max_tokensinteger nullable

Maximum number of tokens to return in the content. If not set, returns full content of the documents.

Example request

{
  "context_filter": {
    "collection": "collection_name",
    "artifacts": [
      "artifact_id_1, artifact_id_2"
    ],
    "metadata_filter": [
      {
        "key": "file_id",
        "operator": "==",
        "value": "artifact_id_1"
      }
    ]
  }
}

Response

Successful chunked content retrieval

Example response

{
  "data": [
    {
      "artifact_id": "annual_report_2023",
      "content": [
        {
          "text": "ANNUAL REPORT 2023\n\nExecutive Summary\n\nFiscal year 2023 marked a transformative period...",
          "type": "text"
        }
      ]
    }
  ]
}

Changes

Changed in 3 of the 11 revisions of this API.615

  • 594fa71dfb90612See the full diff
    • the response property data/items/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source/oneOf[subschema #2: PlainTextSource]/type became optional for the status 200

      response-property-became-optional

    • the response property data/items/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source/oneOf[subschema #3: ContentSource]/type became optional for the status 200

      response-property-became-optional

    • the response property data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source/oneOf[subschema #2: PlainTextSource]/type became optional for the status 200

      response-property-became-optional

    • the response property data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source/oneOf[subschema #3: ContentSource]/type became optional for the status 200

      response-property-became-optional

    • added Base64Source URLDocumentSource to the data/items/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property oneOf list for the response status 200

      response-property-one-of-added

    • added Base64Source URLDocumentSource to the data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property oneOf list for the response status 200

      response-property-one-of-added

    • added LocalResourceBlock to the data/items/content/items/ response property anyOf list for the response status 200

      response-property-any-of-added

    • added LocalResourceBlock to the data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/ response property anyOf list for the response status 200

      response-property-any-of-added

    • the type response's property default value content was added for the status 200

      response-property-default-value-added

    • the type response's property default value content was added for the status 200

      response-property-default-value-added

    • the type response's property default value text was added for the status 200

      response-property-default-value-added

    • the type response's property default value text was added for the status 200

      response-property-default-value-added

    • mapped value for discriminator key base64 changed from Base64PDFSource to Base64Source for data/items/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property for the response status 200

      response-property-discriminator-mapping-changed

    • mapped value for discriminator key base64 changed from Base64PDFSource to Base64Source for data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property for the response status 200

      response-property-discriminator-mapping-changed

    • mapped value for discriminator key url changed from URLPDFSource to URLDocumentSource for data/items/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property for the response status 200

      response-property-discriminator-mapping-changed

    • mapped value for discriminator key url changed from URLPDFSource to URLDocumentSource for data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property for the response status 200

      response-property-discriminator-mapping-changed

    • removed Base64PDFSource URLPDFSource from the data/items/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property oneOf list for the response status 200

      response-property-one-of-removed

    • removed Base64PDFSource URLPDFSource from the data/items/content/items/anyOf[subschema #16: TLDRBlock -> subschema #17: TLDRBlock]/content/items/anyOf[subschema #13: DocumentBlock -> subschema #14: DocumentBlock]/source response property oneOf list for the response status 200

      response-property-one-of-removed

    This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

    • added MidConvSystemBlock to the data/items/content/items/ response property anyOf list for the response status 200

      response-property-any-of-added

    • added MidConvSystemBlock to the data/items/content/items/anyOf[subschema #15: TLDRBlock -> subschema #16: TLDRBlock]/content/items/ response property anyOf list for the response status 200

      response-property-any-of-added

    • endpoint added

      endpoint-added

    This revision also has 13 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog