Clusters

Create Cluster

Changed on

Create a new cluster configuration and output collection.

This endpoint:
1. Creates cluster metadata
2. Creates output collection for cluster documents
3. Returns cluster metadata with output_collection_id

The cluster can then be executed via POST /v1/clusters/{id}/execute
post/v1/clusters

Request

  • Base URL: https://api.mixpeek.com
  • URL: https://api.mixpeek.com/v1/clusters
  • Auth: HTTP bearer and API key in header X-Namespace

Request body

collection_idsstring[] nullable

Collections to cluster together. Required unless source_cluster_ids is set, in which case it defaults to the source clusters' input collections.

source_cluster_idsstring[] nullable

Makes this a composite cluster: it groups the centroids of these clusters' runs instead of documents. Each id must be a vector cluster in this namespace with the same feature_uris as this one. Executing a composite cluster without a mode runs mode 'composite' over the newest completed run of each source cluster, one per partition (a partition is a run's filters). Vector clusters only.

cluster_namestring nullable

Optional human-friendly name for the clustering job

descriptionstring nullable

Optional description of the cluster, stored with it

metadataobject nullable

Optional caller metadata, stored with the cluster

cluster_type'vector' | 'attribute'

Type of clustering to perform.

Determines the clustering approach:

  • vector: Cluster documents by embedding similarity (semantic clustering)
  • attribute: Cluster documents by metadata attributes (business logic clustering)

Use Cases: vector: - Group semantically similar content - Find content with similar meaning - Organize by topic/theme - Requires vector embeddings

attribute:
    - Group by business attributes (category, brand, status, etc.)
    - Organize by explicit metadata
    - Create hierarchical groupings
    - No embeddings required
enrich_source_collectionboolean

If True, cluster results are written back to source collection(s) in-place instead of creating new output collections. Documents will be enriched with cluster_id, cluster_label, distance_to_centroid, and optionally other metadata. Similar to taxonomy enrichment pattern.

auto_execute_on_batchboolean

Automatically execute this cluster whenever a batch completes on any of its input collections. When True, a ClusterApplicationConfig entry is added to each input collection's cluster_applications field at creation time. The cluster will then auto-trigger after each batch completion (subject to cooldown and document threshold). When False (default), the cluster must be executed manually via the API.

auto_execute_min_documentsinteger nullable

Minimum number of documents required before auto-executing cluster. Only used when auto_execute_on_batch=True. If the collection has fewer documents than this threshold, clustering is skipped.

auto_execute_cooldown_secondsinteger

Minimum time (in seconds) between automatic cluster executions. Only used when auto_execute_on_batch=True. Default: 3600 (1 hour).

Example request

{
  "cluster_name": "products_clip_hdbscan",
  "cluster_type": "vector",
  "collection_ids": [
    "col_products_v1",
    "col_products_v2"
  ],
  "llm_labeling": {
    "enabled": true,
    "labeling_inputs": {
      "input_mappings": [
        {
          "input_key": "text",
          "path": "description",
          "source_type": "payload"
        }
      ]
    },
    "model_name": "gpt-4o-mini-2024-07-18",
    "provider": "openai"
  },
  "vector_config": {
    "clustering_method": "hdbscan",
    "feature_uri": "mixpeek://clip_vit_l_14@v1/embedding",
    "hdbscan_parameters": {
      "min_cluster_size": 10,
      "min_samples": 5
    },
    "sample_size": 5000
  }
}

Response

Successful Response

cluster_idstring

Unique cluster job identifier

cluster_namestring required

Human-readable cluster name

descriptionstring nullable

Cluster description

namespace_idstring required

Namespace this cluster belongs to

input_collectionsstring[] required

Source collection IDs that were clustered

source_cluster_idsstring[] nullable

Set on a composite cluster: the clusters whose run centroids it groups. Executing it without a mode runs mode 'composite' over the newest completed run of each source cluster, one per partition.

source_bucket_idsstring[] nullable

Source bucket IDs that the input collections originated from. Enables bucket lineage tracking.

filtersobject nullable

Optional filters that were applied to pre-filter documents before clustering

cluster_type'vector' | 'attribute' required

Type of clustering: vector (embedding-based) or attribute (metadata-based)

feature_urisstring[] nullable

Feature URIs that were clustered (mixpeek://{extractor}@{version}/{output}). Only for vector clustering.

multi_feature_strategystring nullable

Strategy used if multiple features (concatenate/independent/weighted). Only for vector clustering.

learned_weightsobject nullable

Automatically learned feature weights (when multi_feature_strategy='weighted'). Keys are feature URIs, values are learned weights. Only populated after clustering execution completes.

learning_quality_scorenumber nullable

Clustering quality score from weight learning (e.g., silhouette score). Only populated when multi_feature_strategy='weighted' and weights were learned.

effective_feature_methodstring nullable

Method for calculating cluster centroids (mean/median/medoid). Only for vector clustering.

sample_sizeinteger nullable

Stored per-execution document cap. Populated from vector_config.sample_size at cluster creation and replayed into ClusteringConfig on every POST /v1/clusters/{id}/execute so re-runs stay consistent with the original config. When None, the export is uncapped (bounded by the engine's 100,000 safety limit). Only applies to vector clustering.

preprocessing_stepsobject[] nullable

Stored preprocessing steps from vector_config. Replayed into ClusteringConfig on every execute.

hierarchical_vectorboolean nullable

Whether recursive sub-clustering is enabled for vector clustering.

max_hierarchy_depthinteger nullable

Maximum recursion depth for hierarchical sub-clustering.

vis_n_components2 | 3 nullable

Stored visualization dimensionality (2D or 3D). Replayed into ClusteringConfig on every execute as the default.

layout_stability'none' | 'transform' | 'align' nullable

Stored layout-stability mode from vector_config.layout_stability. Replayed into ClusteringConfig on every execute. When unset, executions default to 'align' (keep the map stable across runs via post-hoc registration).

clustered_attributesstring[] nullable

Attribute field names that were clustered. Only for attribute clustering.

hierarchical_groupingboolean nullable

Whether hierarchical clustering was used. Only for attribute clustering.

aggregation_methodstring nullable

Method for aggregating attributes (most_frequent/first/last). Only for attribute clustering.

output_collection_idsstring[]

Collection IDs where cluster documents are stored. Each holds one centroid document (is_centroid=true) per cluster and one member document (is_centroid=false) per clustered source document. A member carries cluster_id, cluster_label and source_document_id, the ID of the document you clustered, so filter on source_document_id to find a source document's cluster. For single output: list with one collection ID. For per-feature output: list with one collection ID per feature. With enrich_source_collection=true no output collection is created: this lists the source collections, and each source document carries its assignment as fields.

output_collection_namesstring[]

Names of output collections. Corresponds to output_collection_ids.

algorithmstring nullable

Clustering algorithm used (hdbscan, kmeans, attribute_based, etc.)

algorithm_paramsobject nullable

Algorithm-specific parameters (not used for attribute_based)

enrich_sourceboolean

Whether source documents were enriched with cluster_id

num_clustersinteger nullable

Number of clusters found (excludes noise/outliers, populated after execution)

num_documents_clusteredinteger nullable

Total documents processed

execution_time_secondsnumber nullable

Time taken to complete clustering

quality_metricsobject nullable

Clustering quality metrics (silhouette_score, davies_bouldin_score, calinski_harabasz_score, etc.). Open diagnostic bag like ClusteringResult.metrics: mostly numeric, but the three-state degenerate flag (False | reason string) and its prose degenerate_detail are non-numeric, so the value type is Any. A float-only type 400'd every GET on a cluster whose run flagged a degenerate result ('s read-path sibling: the write side persisted the string, this read side then rejected it).

hierarchy_detectedboolean

Whether implicit hierarchy was detected (multi-feature independent) or created (hierarchical attributes)

parent_cluster_idstring nullable

For child clusters in hierarchy

child_cluster_idsstring[] nullable

For parent clusters

hierarchy_relationshipsobject[] nullable

Parent-child relationships detected from cluster membership overlap

status'PENDING' | 'QUEUED' | 'IN_PROGRESS' | 'PROCESSING' | 'COMPLETED' | 'COMPLETED_WITH_ERRORS' | 'FAILED' | 'CANCELED' | 'INTERRUPTED' | 'UNKNOWN' | 'SKIPPED' | 'DRAFT' | 'ACTIVE' | 'ARCHIVED' | 'SUSPENDED' | 'DEACTIVATED'

Enumeration of task statuses for tracking asynchronous operations.

Task statuses indicate the current state of asynchronous operations like batch processing, object ingestion, clustering, and taxonomy execution.

Status Categories: Operation Statuses: Track progress of async operations Lifecycle Statuses: Track entity state (buckets, collections, namespaces)

Values: PENDING: Task is queued but has not started processing yet IN_PROGRESS: Task is currently being executed PROCESSING: Task is actively processing data (similar to IN_PROGRESS) COMPLETED: Task finished successfully with no errors COMPLETED_WITH_ERRORS: Task finished but some items failed (partial success) FAILED: Task encountered an error and could not complete CANCELED: Task was manually canceled by a user or system UNKNOWN: Task status could not be determined SKIPPED: Task was intentionally skipped DRAFT: Task is in draft state and not yet submitted

ACTIVE: Entity is active and operational (for buckets, collections, etc.)
ARCHIVED: Entity has been archived
SUSPENDED: Entity has been temporarily suspended

Terminal Statuses: COMPLETED, COMPLETED_WITH_ERRORS, FAILED, CANCELED are terminal statuses. Once a task reaches these states, it will not transition to another state.

Partial Success Handling: COMPLETED_WITH_ERRORS indicates that the operation completed but some documents/items failed. The task result includes: - List of successful items - List of failed items with error details - Success rate percentage This allows clients to handle partial success scenarios appropriately.

Polling Guidance: - Poll tasks in PENDING, QUEUED, IN_PROGRESS, or PROCESSING states - Stop polling when task reaches COMPLETED, COMPLETED_WITH_ERRORS, FAILED, or CANCELED - Use exponential backoff (1s → 30s) when polling

errorstring nullable

Error message if cluster execution failed. Propagated from TaskService.

failure_category'timeout' | 'infrastructure' | 'orphaned' | 'pipeline' | 'mvs_durability' | 'validation' | 'unknown'

Batch-level failure classification.

Coarser-grained than ErrorCategory (which classifies individual object errors). FailureCategory is set on the batch itself to tell users why the batch as a whole failed — timeout, infra, orphan, pipeline, or unknown. Drives the "Batch failed: <category>" badge in Studio and lets callers distinguish retryable infra blips from genuine pipeline bugs without parsing human-readable strings.

last_execution_task_idstring nullable

Most recent task ID for this cluster

last_run_idstring nullable

Most recent execution run ID

created_atstring date-time

When cluster was created

updated_atstring date-time

When cluster was last updated

last_executed_atstring date-time nullable

Last execution timestamp

completed_atstring date-time nullable

When clustering completed successfully

llm_labeling_errorsstring[] nullable

List of errors encountered during LLM labeling (if any). Stored in MongoDB cluster metadata only, NOT in vector store cluster documents. Used to track LLM failures while allowing fallback labels to work.

metadataobject

Additional user-defined metadata

Example response

{
  "source_enrichment_config": {
    "field_mappings": [
      {
        "source_field": "cluster_id",
        "target_field": "category_id"
      },
      {
        "source_field": "cluster_label",
        "target_field": "category_name"
      },
      {
        "source_field": "distance_to_centroid",
        "target_field": "category_confidence"
      }
    ]
  },
  "llm_labeling": {
    "description": "Text-only labeling with multiple fields",
    "enabled": true,
    "include_keywords": true,
    "include_summary": true,
    "labeling_inputs": {
      "input_mappings": [
        {
          "input_key": "title",
          "path": "title",
          "source_type": "payload"
        },
        {
          "input_key": "description",
          "path": "description",
          "source_type": "payload"
        },
        {
          "input_key": "text",
          "path": "text",
          "source_type": "payload"
        }
      ]
    },
    "model_name": "gpt-4o-mini-2024-07-18",
    "provider": "openai"
  }
}

Changes

  • 24540a9a39c0Revision changes
    • info

      added the new optional request property

    • info

      the request property became optional

    • info

      request property list-of-types was widened by adding types null to media type application/json

    • info

      added the optional property to the response with the status

  • 3516cddef582Revision changes
    • warning

      added the new gemini-3.5-flash-lite enum value to the /// response property for the response status

    • info

      added the new gemini-3.5-flash-lite enum value to the request property ///