Analytics

Get Crowd Bt

CrowdBT joint fit — entity scores + annotator reliability η.

Chen et al. 2013, "Pairwise Ranking Aggregation in a Crowdsourced Setting." Each annotator k gets a learned reliability parameter η_k ∈ [0, 1] alongside the BT entity scores. Versus AimRank's operational κ-weighting (Layer 4): same goal, fitted jointly via L-BFGS-B instead of computed post-hoc against majority consensus.

Use this when:

  • You want a principled per-annotator quality signal (not threshold- based) for downstream κ-weighted DPO training
  • Your data has identifiable annotators with ≥ min_votes_per_annotator votes each
  • Anti-correlated raters (η < 0.5) need to surface explicitly, not collapse into "noisy"

Falls back to converged=false with a reason when data is too sparse — see services/crowd_bt.py for the failure modes.

get/api/v1/analytics/rankings/{ranking_id}/crowd-bt

Request

  • Base URL: https://app.aimrank.io (the document declares no server; this is the origin it was published from)
  • URL: https://app.aimrank.io/api/v1/analytics/rankings/{ranking_id}/crowd-bt
  • Auth: none declared

Path parameters

ranking_idstring required

Query parameters

min_votes_per_annotatorinteger
min_votes_per_entityinteger
max_iterationsinteger

Response

Successful Response

unknown required

Changes

No recorded changes to this operation across all 1 revision of this API.