---
title: "Evaluate parsed transactions against benchmark ground truth"
method: POST
path: "/benchmark/evaluate"
tags: ["Benchmark"]
---

# Evaluate parsed transactions against benchmark ground truth

`POST /benchmark/evaluate`

## Request

- Base URL: `https://api.bankstatemently.com/v1`
- URL: `https://api.bankstatemently.com/v1/benchmark/evaluate`
- Auth: API key in header `X-API-Key`

Submit parsed transactions from a [Bank Statement Parsing Benchmark](https://github.com/bankstatemently/bank-statement-parsing-benchmark) PDF for scoring. The `contentHash` is the SHA-256 hex digest of the PDF you parsed — this ties the evaluation to the exact file. Free to use, no credits consumed. Rate limit: 50 requests per hour.

## Request body

- object
  - `contentHash` string, required: SHA-256 hex digest of the benchmark PDF file.
  - `accounts` object[]: Optional account roster for multi-account statements. Each transaction references one via accountId.
    - `id` string, required: Submission-internal handle, referenced by transactions[].accountId.
    - `accountNumber` string: Verbatim printed account number — never normalized by the submitter.
    - `name` string: Verbatim printed account label.
    - `currency` string: ISO 4217 currency code.
  - `transactions` BenchmarkTransaction[], required: Parsed transactions from the benchmark statement (max 2000).
    - `date` string, required: ISO 8601 date (YYYY-MM-DD).
    - `description` string, required: Transaction description.
    - `amount` number, required: Transaction amount. Positive absolute value, or negative to indicate debit. When direction is also provided, positive amounts are treated as absolute values.
    - `direction` 'credit' | 'debit': Explicit direction. If omitted, inferred from amount sign (negative = debit, positive/zero = credit). Negative amount with direction "credit" is rejected as contradictory.
    - `balance` number: Running balance after this transaction.
    - `currency` string: ISO 4217 currency code. Defaults to statement currency.
    - `originalData` object, required: Raw column values as they appear in the PDF.

## Response `200`

Evaluation results with both normalized and parsed scores.

- BenchmarkEvaluationResult
  - `contentHash` string, required: Echo of the submitted content hash.
  - `id` string, required: Statement ID (e.g. bsb-001).
  - `datasetVersion` string, required: Ground truth dataset version.
  - `difficulty` 'basic' | 'intermediate' | 'advanced', required: Statement difficulty tier.
  - `challenges` string[], required: Parsing challenges present in the statement.
  - `normalizedScore` BenchmarkScoreDetail, required
    - `overall` number: Headline score: extraction × integrity (0-1).
    - `extraction` number: Weighted accuracy across all fields (0-1).
    - `integrity` number: Structural correctness: validation checks, row coverage, direction accuracy (0-1).
    - `fields` object: Per-field accuracy scores. Normalized uses canonical names (date, description, amount). Parsed uses original column headers from the PDF.
    - `alignment` object
      - `matched` integer: Transactions matched between submission and ground truth.
      - `missing` integer: Transactions in ground truth but not in submission.
      - `extra` integer: Transactions in submission but not in ground truth.
      - `total` integer: Total ground truth transactions.
  - `parsedScore` BenchmarkScoreDetail, required
    - `overall` number: Headline score: extraction × integrity (0-1).
    - `extraction` number: Weighted accuracy across all fields (0-1).
    - `integrity` number: Structural correctness: validation checks, row coverage, direction accuracy (0-1).
    - `fields` object: Per-field accuracy scores. Normalized uses canonical names (date, description, amount). Parsed uses original column headers from the PDF.
    - `alignment` object
      - `matched` integer: Transactions matched between submission and ground truth.
      - `missing` integer: Transactions in ground truth but not in submission.
      - `extra` integer: Transactions in submission but not in ground truth.
      - `total` integer: Total ground truth transactions.

## Other responses

- `400` (unresolved $ref)
- `401`: Authentication failed, API key revoked, or API key expired.
- `429`: Too many requests. Wait the number of seconds indicated by the `Retry-After` header.
- `500`: An unexpected server error occurred.

---

[API](https://skmtc.dev/bankstatemently/apis/bankstatemently-api.md) · [All operations](https://skmtc.dev/bankstatemently/apis/bankstatemently-api/llms.txt) · [OpenAPI document](https://skmtc.dev/bankstatemently/apis/bankstatemently-api/revisions/4c3b7fb26edf?raw)