Inference

Benchmark on-disk text-generation models

Benchmarks every text-generation model on disk against a fixed prompt set, producing per-model latency/throughput rows. When a disclosed, score-only leaderboard egress is configured, anonymous result rows are auto-uploaded to the public ModelPiper leaderboard (no prompts or content leave the device). Only one benchmark can run at a time (429 if already in progress).

post/v1/benchmark/llm

Request

  • Base URL: http://localhost:9998
  • URL: http://localhost:9998/v1/benchmark/llm
  • Auth: HTTP bearer

Response

Benchmark completed; per-model results and upload tally

countinteger required

Number of models benchmarked

resultsobject[] required

Per-model benchmark rows (WireBenchmarkSubmission)

skippedobject[] required

Models skipped (with reason)

uploadedinteger required

Rows uploaded to the leaderboard

uploadFailedinteger required

Rows that failed to upload

Changes

No recorded changes to this operation across all 1 revision of this API.