Utilities

Segment an image Asset into per-region masks (non-minting)

The segment API's non-minting REST door (the image_segment MCP verb's sibling): runs the SAME capability-checked segmentation seam — the user-selectable model registry (Apple Vision foreground-instance floor by default, per-call override via model) with typed rejections for an unknown model id or a prompt mode the selected entry does not declare — but writes NO Asset. Every produced region mask returns inline as base64 PNG, primary first (highest selection score: prompt coverage for point/box, area fraction for auto on the Apple Vision floor, the model's own mask confidence for auto on a SAM3-class entry — where that ordering carries no claim about which region matters). Minting stays verb-only: call the image_segment tool to record mask Assets with provenance.

post/v1/image/segment

Request

  • Base URL: http://localhost:9998
  • URL: http://localhost:9998/v1/image/segment
  • Auth: HTTP bearer

Request body

assetstring required

Id of the input image Asset to segment.

mode'auto' | 'point' | 'box' | 'text'

Segmentation mode: text (every instance matching a noun-phrase concept — requires a text-capable segmentation model, a typed 400 otherwise; the concept must be a CONCRETE noun such as person or red car, since abstract prompts like object, subject or foreground match nothing), point (instances covering the given points), box (instances overlapping the rectangle), or auto (no prompt: foreground instances on the Apple Vision floor, and on SAM3-class entries an EXHAUSTIVE DECOMPOSITION via automatic mask generation over a dense point grid. It partitions the scene into regions of everything and frequently does not contain the subject at all, returned in no meaningful order — the score is the model's confidence in its own mask, which favours large flat areas and is a poor proxy for importance. Use auto to enumerate a scene and consume the whole set; prefer text / point / box to find a subject.)

textstring

Noun-phrase concept for text mode (e.g. red car).

modelstring

Optional segmentation model id for this call, overriding the segmentationModelId setting (default vision-foreground-instance). An unknown id is a typed 400 naming the registered entries, never a silent fallback.

Response

Per-region masks, primary first; nothing is written to the Asset graph.

countinteger required

Number of region masks returned.

widthinteger required

Source image pixel width.

heightinteger required

Source image pixel height.

mode'auto' | 'point' | 'box' | 'text' required
modelstring required

The segmentation model id that produced the masks.

Changes

No recorded changes to this operation across all 1 revision of this API.