---
title: "Evaluation Ai Human Agreement"
method: GET
path: "/core/evaluations/{evaluation_id}/agreement"
tags: ["Evaluations"]
---

# Evaluation Ai Human Agreement

`GET /core/evaluations/{evaluation_id}/agreement`

## Request

- Base URL: `https://api.anyreach.ai` (the document declares no server; this is the origin it was published from)
- URL: `https://api.anyreach.ai/core/evaluations/{evaluation_id}/agreement`
- Auth: HTTP bearer

Ad-hoc AI-vs-human agreement over this evaluation's EXISTING runs.

Pairs each conversation's AI verdict against its human verdict per
criterion and returns Cohen's κ + percent agreement — the same κ math as
a calibration set's Gate 2, computed on-the-fly over the runs that
already exist (no calibration set required).

DELIBERATE divergence from a formal calibration set: this takes the
single MOST-RECENT run per (conversation, assessor) rather than pooling
every assignment into a consensus, so the two can differ when a
conversation has multiple runs by the same assessor. `na` outcomes are
excluded from κ (an N/A is "doesn't apply", not a disagreement); for a
rigorous, multi-rater, applicability-aware measure, use a calibration set.

## Path parameters

- `evaluation_id` string, required

## Headers

- `x-anyreach-org` string, nullable: Organization ID for user PATs (pat_ prefix tokens)

## Response `200`

Successful Response

- unknown

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/anyreach/apis/anyreach-api.md) - [All operations](https://skmtc.dev/anyreach/apis/anyreach-api/llms.txt) - [OpenAPI document](https://skmtc.dev/anyreach/apis/anyreach-api/revisions/2eddf40c8c93?raw)