Sign in

SDPM Chunker

Extends semantic chunking using a double-pass merging approach (Semantic Double-Pass Merging). Connects related content even if not consecutive.

post/v1/chunk/sdpm

Response

Successful Response: A list of SemanticChunk objects.

textstring

The actual text content of the chunk.

start_indexinteger

The starting character index of the chunk within the original input text.

end_indexinteger

The ending character index (exclusive) of the chunk within the original input text.

token_countinteger

The number of tokens in this specific chunk, according to the tokenizer used.

Changes

Changed in 11 of the 20 revisions of this API.141526

  • e12d0b1f23dc11See the full diff
    • ●

      removed the request property skip_window

      request-property-removed

    • ○

      added the new optional request property include_delim

      new-optional-request-property

  • bc64e17759521See the full diff
    • ○

      endpoint added

      endpoint-added

    This revision also has 1 change that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • 5c8e448048061See the full diff
    • ▲

      api removed without deprecation

      api-removed-without-deprecation

    This revision also has 19 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • 1762a3ecb81b2See the full diff
    • ○

      the delim request property default value changed from . ! ? to . ! ?

      request-property-default-value-changed

    • ○

      the min_characters_per_sentence request property default value changed from 1 to 12

      request-property-default-value-changed

  • 7e3cb743a9ed1See the full diff
    • ●

      removed the request property text

      request-property-removed

  • b5fe83cf70c611See the full diff
    • ▲

      removed the media type application/json from the request body

      request-body-media-type-removed

    • ○

      added the media type multipart/form-data to the request body

      request-body-media-type-added

  • 502b267e607d12See the full diff
    • ▲

      request property return_type was restricted to a list of enum values

      request-property-became-enum

    • ○

      added the new chunks enum value to the request property return_type

      request-property-enum-value-added

    • ○

      added the new texts enum value to the request property return_type

      request-property-enum-value-added

  • 2d174e4e5a5a1See the full diff
    • ○

      the request property chunk_size became optional

      request-property-became-optional

  • a9d5b6b906759111See the full diff
    • ▲

      the response property items/end_index became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/sentences became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/sentences/items/end_index became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/sentences/items/start_index became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/sentences/items/text became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/sentences/items/token_count became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/start_index became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/text became optional for the status 200

      response-property-became-optional

    • ▲

      the response property items/token_count became optional for the status 200

      response-property-became-optional

    • ●

      removed the request property args

      request-property-removed

    • ○

      added the new optional request property delim

      new-optional-request-property

    • ○

      added the new optional request property embedding_model

      new-optional-request-property

    • ○

      added the new optional request property min_characters_per_sentence

      new-optional-request-property

    • ○

      added the new optional request property min_sentences

      new-optional-request-property

    • ○

      added the new optional request property mode

      new-optional-request-property

    • ○

      added the new optional request property return_type

      new-optional-request-property

    • ○

      added the new optional request property similarity_window

      new-optional-request-property

    • ○

      added the new optional request property skip_window

      new-optional-request-property

    • ○

      added the new optional request property threshold

      new-optional-request-property

    • ○

      added the new optional request property threshold_step

      new-optional-request-property

    • ○

      added the new required request property chunk_size with a default value

      new-required-request-property-with-default

  • 678e906303364See the full diff
    • ○

      removed the non-success response with the status 401

      response-non-success-status-removed

    • ○

      removed the non-success response with the status 422

      response-non-success-status-removed

    • ○

      removed the non-success response with the status 500

      response-non-success-status-removed

    • ○

      added the required property items/sentences to the response with the 200 status

      response-required-property-added