---
title: "Advisory memory verdicts for several models"
method: POST
path: "/v1/models/fit/batch"
tags: ["Models"]
---

# Advisory memory verdicts for several models

`POST /v1/models/fit/batch`

## Request

- Base URL: `http://localhost:9998`
- URL: `http://localhost:9998/v1/models/fit/batch`
- Auth: HTTP bearer

Advisory verdicts for up to 32 models against ONE host budget reading, so
every answer in a comparison agrees about how much memory the machine
has. Results come back in request order.

A request whose `modelId` names no model on this machine or in the
bundled catalog is omitted from `results` rather than failing the batch.
A batch of more than 32 requests is refused with 400 rather than
truncated, because a truncated batch reads as a ranking. An empty batch
is also a 400. Nothing is reserved, as with `POST /v1/models/fit`.

## Request body

- WireFitVerdictRequest[]
  - `modelId` string, required
  - `allocatedTokens` integer, nullable: KV capacity in tokens the backend allocates. Not the tokens a request prepares.
  - `streams` integer, nullable: Concurrent sequences sharing the model. KV scales with this.
  - `keyElementBytes` integer, nullable: Bytes per stored key element (2 for F16, 1 for an 8-bit KV cache).
  - `valueElementBytes` integer, nullable: Bytes per stored value element.
  - `mode` string, nullable: Where the allocation lives: `fullGpu` or `cpuOffload`. Tolerate unknown values.

## Response `200`

Advisory verdicts, in request order

- WireFitVerdictBatch: A bounded batch of advisory answers, in request order.
  - `results` WireFitVerdictResult[], required
    - `modelId` string, required
    - `identity` WireFitEvidenceIdentity, required: What the answer is about, precisely enough that a later answer about a different artifact is recognisable.
      - `modelId` string, required
      - `revision` string, required: Immutable commit the bytes were fetched at; empty when the source recorded none.
      - `quant` string, required: Empty when the artifact declares none.
      - `artifactBytes` integer, nullable, required: Measured bytes of the evaluated file set; null when not on disk.
    - `estimate` WireFitMemoryEstimate, required: A complete memory forecast. `lowerBoundBytes` is always a real floor; `upperBoundBytes` is finite only when no part is unknown.
      - `parts` WireFitMemoryPart[], required
        - `component` string, required: `weights`, `kvCache`, `recurrentState`, `workspace` or `companions`. Tolerate unknown values.
        - `bytes` integer, nullable, required: Bytes; null if and only if `basis` is `unknown`.
        - `basis` string, required: `measuredArtifact`, `derivedLayout`, `parameterProxy`, `notApplicable` or `unknown`. Tolerate unknown values.
        - `sharedKey` string, required: Identity of an allocation shared with other models in the same evaluation; empty when not shared.
        - `note` string, required: Why this basis, in the producer's words.
      - `lowerBoundBytes` integer, required: Sum of every part that carries a number. A floor, never the answer.
      - `upperBoundBytes` integer, nullable, required: Null means the total is unknown.
      - `weakestBasis` string, required: The weakest basis among the parts; the total is no stronger than this. Tolerate unknown values.
      - `missing` string[], required: Components that would have to be observed for `upperBoundBytes` to be finite.
      - `workload` WireFitWorkload, required: The workload actually evaluated, after any documented-default capacity cap.
        - `allocatedTokens` integer, required
        - `contextSource` string, required: `documentedDefault` (the product default, which may be capped to the model limit) or `requested` (a caller named it; over-limit is refused, never truncated). Tolerate unknown values.
        - `streams` integer, required
        - `keyElementBytes` integer, required
        - `valueElementBytes` integer, required
        - `mode` string, required: `fullGpu` or `cpuOffload`. Tolerate unknown values.
        - `preparedTokens` integer, nullable, required: Tokens the request actually prepares; null until an exact prepared count is available. Never derived from capacity.
      - `refusal` string, required: `none`, or why no estimate was produced: `contextExceedsModelLimit`, `nonPositiveContext`, `nonPositiveStreams`, `nonFiniteInput`. Tolerate unknown values.
    - `verdict` string, required: `good` (the ceiling fits with the presentation margin), `tight` (fits below it), `oversize` (the floor already exceeds the budget) or `unknown`. Tolerate unknown values.
    - `budget` WireFitBudget, required: The budget one verdict was taken against, as it was read.
      - `basis` string, required: `hostAvailable` (what a load could claim right now), `hostTotal` (the physical ceiling) or `unknown`. Tolerate unknown values.
      - `bytes` integer, nullable, required: Bytes; null if and only if `basis` is `unknown`.
      - `committedBytes` integer, nullable, required: Bytes already committed to loaded models when the budget was read.
    - `gpuAdmits` boolean, nullable, required: Whether GPU admission would admit the estimate's floor without evicting anything; null when no GPU answer was obtainable. Not a reservation.
    - `limits` string[], required: Evidence the estimator could not use for this answer: `artifactNotOnDisk`, `layoutUnpublished`, `layoutNotUniform`, `historyIdentityInsufficient`, `workspaceUnmeasured`, `preparedCountUnavailable`, `hostTelemetryUnavailable`, `gpuTelemetryUnavailable`. Tolerate unknown values.

## Other responses

- `400`: Invalid request
- `499`: The batch was cancelled before it finished (`type` is `cancelled`)
- `500`: The batch failed

---

[API](https://skmtc.dev/modelpiper/apis/toolpiper-api.md) · [All operations](https://skmtc.dev/modelpiper/apis/toolpiper-api/llms.txt) · [OpenAPI document](https://skmtc.dev/modelpiper/apis/toolpiper-api/revisions/6722bf56385c?raw)