---
title: "Benchmark on-disk text-generation models"
method: POST
path: "/v1/benchmark/llm"
tags: ["Inference"]
---

# Benchmark on-disk text-generation models

`POST /v1/benchmark/llm`

## Request

- Base URL: `http://localhost:9998`
- URL: `http://localhost:9998/v1/benchmark/llm`
- Auth: HTTP bearer

Benchmarks every text-generation model on disk against a fixed prompt set,
producing per-model latency/throughput rows. When a disclosed, score-only
leaderboard egress is configured, anonymous result rows are auto-uploaded to
the public ModelPiper leaderboard (no prompts or content leave the device).
Only one benchmark can run at a time (429 if already in progress).

## Response `200`

Benchmark completed; per-model results and upload tally

- object
  - `count` integer, required: Number of models benchmarked
  - `results` object[], required: Per-model benchmark rows (WireBenchmarkSubmission)
  - `skipped` object[], required: Models skipped (with reason)
  - `uploaded` integer, required: Rows uploaded to the leaderboard
  - `uploadFailed` integer, required: Rows that failed to upload

## Other responses

- `400`: Unsupported hardware (not Apple Silicon) or no models on disk
- `428`: Leaderboard egress is not configured or not enabled
- `429`: A benchmark is already in progress

---

[API](https://skmtc.dev/modelpiper/apis/toolpiper-api.md) · [All operations](https://skmtc.dev/modelpiper/apis/toolpiper-api/llms.txt) · [OpenAPI document](https://skmtc.dev/modelpiper/apis/toolpiper-api/revisions/6722bf56385c?raw)