Parse text

Splits text based on token count. Best for ensuring chunks fit within LLM context windows and maintaining consistent chunk sizes for embedding models.

Splits text at sentence boundaries while respecting token limits. Ideal for maintaining semantic coherence and readability.

Recursively splits text using a hierarchy of separators (paragraphs, sentences, words). Versatile general-purpose chunker that preserves document structure.

Groups semantically similar sentences using embeddings. Excellent for maintaining topic coherence and context within chunks.

Agentic LLM-powered chunker that uses AI to determine optimal split points. Best for complex documents requiring intelligent segmentation.

High-performance SIMD-optimized byte-level chunking. Best for large files (>1MB) where speed and memory efficiency are critical. 2x faster and 3x less memory than token-based chunking.

Splits text after preserving the full-document context in its embeddings. Best for retrieval workflows where each chunk should retain information from the surrounding document.