---
title: "Run one declared benchmark workload"
method: POST
path: "/v1/benchmark/harness"
tags: ["Inference"]
---

# Run one declared benchmark workload

`POST /v1/benchmark/harness`

## Request

- Base URL: `http://localhost:9998`
- URL: `http://localhost:9998/v1/benchmark/harness`
- Auth: HTTP bearer

Runs one cold and 1 to 32 warm iterations of a declared workload against
the real engine, or of the empty harness adapter. Stores private
observations locally and returns only the public field allowlist. No
observation is uploaded. Device, machine, run and attempt identifiers
are excluded from every response.

## Request body

- object
  - `workload` string, required: `harness-floor` times an empty adapter and loads no model. Any other declared workload (see `GET /v1/benchmark/workloads`) measures that engine family against a fixed synthetic input; the cold iteration may include the model load.
  - `repetitions` integer, required

## Response `200`

Separate cold and warm observations, one per repetition

- BenchPublicObservation[]
  - `version` 1, required
  - `workload` object, required
    - `id` string, required
    - `revision` string, required
    - `inputRevision` string, required
    - `engineId` 'piper-engine' | 'piper-tts' | 'piper-stt' | 'piper-mlx-tts' | 'piper-mlx-llm' | 'apple-intelligence' | 'apple-vision-ocr' | 'embedding-server' | 'embedding-gemma' | 'coreml-upscale' | 'coreml-pose' | 'image-generation' | 'video-generation' | 'mesh-generation' | 'splat-generation' | 'cloud' | 'harness', required
    - `modelId` string, required
    - `modelRevision` string, nullable, required
    - `runtimeRevision` string, nullable, required
    - `methodId` string, required
    - `contextTokens` integer, nullable, required
    - `outputLimit` integer, nullable, required
    - `keyCacheType` string, nullable, required
    - `valueCacheType` string, nullable, required
    - `batch` integer, nullable, required
    - `microBatch` integer, nullable, required
    - `cachePolicy` string, required
    - `stopPolicy` string, required
    - `speculationPolicy` string, required
    - `suiteRevision` string, nullable, required
    - `ramTierGb` integer, nullable, required
    - `steps` integer, nullable, required
    - `resolution` string, nullable, required
    - `voice` string, nullable, required
  - `osBuild` string, required
  - `appVersion` string, required
  - `buildMode` 'Release' | 'Debug', required
  - `observedAt` string, required
  - `powerSource` 'ac' | 'battery' | 'unknown', required
  - `lowPowerMode` boolean, required
  - `thermalState` 'nominal' | 'fair' | 'serious' | 'critical' | 'unknown', required
  - `provenance` 'measured' | 'estimated' | 'historical' | 'diagnostic', required
  - `stage` 'cold' | 'warm', required
  - `ordinal` integer, required
  - `outcome` 'completed' | 'incomplete' | 'cancelled' | 'failed', required
  - `failure` 'cancelled' | 'adapter-error' | 'early-stop' | 'missing-statistics' | 'invalid-evidence' | 'not-run', nullable, required
  - `elapsedMs` number, required
  - `metrics` object[], required
    - `name` 'prefill' | 'decode' | 'first-content' | 'elapsed' | 'frames' | 'items' | 'audio-realtime' | 'audio-duration' | 'text-encode' | 'denoise' | 'vae-decode' | 'harness-floor' | 'unknown', required
    - `unknownName` string, nullable, required
    - `unit` 'tokens/s' | 'ms' | 'frames/s' | 'items/s' | 'ratio', required
    - `value` number, required
    - `tokens` object, nullable, required
      - `count` object, required
        - `quantity` 'providerUsage' | 'preparedPrompt' | 'generatedSequence' | 'artifact' | 'estimate', required
        - `category` 'input' | 'output' | 'cachedInput' | 'cacheCreationInput' | 'reasoningOutput' | 'total', required
        - `value` integer, nullable, required
        - `relation` 'inclusiveTotal' | 'subset' | 'disjoint', required
        - `derived` boolean, required
        - `contractRef` object, nullable, required
          - `contractId` string, required
          - `assetDigest` string, required
          - `implementationVersion` string, required
          - `modelRevision` string, required
          - `templateDigest` string, nullable, required
          - `encodingOptions` object, required
            - `addSpecialTokens` boolean, required
            - `addBosToken` boolean, required
            - `addEosToken` boolean, required
            - `addGenerationPrompt` boolean, required
            - `normalization` 'none' | 'nfc' | 'nfkc' | 'modelDefault', required
      - `evidence` object, required
        - `method` 'providerReport' | 'runtimeReport' | 'localTokenizer' | 'engineTokenizer' | 'heuristic' | 'unavailable', required
        - `availability` 'complete' | 'partial' | 'missing' | 'unsupported' | 'invalid' | 'mismatch', required
        - `persistence` 'uncommitted' | 'committed' | 'unavailable', required
        - `reconciliation` 'notAttempted' | 'pending' | 'agreed' | 'disagreed' | 'unreconcilable', required
      - `interval` 'processed-prompt' | 'generated-after-first', required
      - `numerator` integer, required
      - `intervalMs` number, required
  - `hardware` object, required
    - `chip` string, required
    - `cpuPCores` integer, required
    - `cpuECores` integer, required
    - `gpuCores` integer, required
    - `ramGb` integer, required
    - `chassis` string, required

## Other responses

- `400`: Unknown workload or invalid repetition count
- `409`: A harness run is already in progress
- `422`: The workload's engine cannot run it now; the body is its capability with the reason
- `500`: Host observation or private ledger persistence failed

---

[API](https://skmtc.dev/modelpiper/apis/toolpiper-api.md) · [All operations](https://skmtc.dev/modelpiper/apis/toolpiper-api/llms.txt) · [OpenAPI document](https://skmtc.dev/modelpiper/apis/toolpiper-api/revisions/6722bf56385c?raw)