CAcaveman

POST /projects/{projectId}/eval-qualification-sets

POST /api/v1/projects/{projectId}/eval-qualification-sets on the Caveman Cloud control plane. Requires the autopilot:manage scope. Not exposed to autonomous agents: Independent holdout labels must not be authored by the agent whose evaluator they qualify.

post/projects/{projectId}/eval-qualification-sets

Request

  • Base URL
    https://api.caveman.so/api/v1
  • URL
    https://api.caveman.so/api/v1/projects/{projectId}/eval-qualification-sets
  • Auth
    • OAuth 2
      Authorization code: /api/v1/oauth/authorize
      Token URL: /api/v1/oauth/token scopes: autopilot:manage
    • or HTTP bearer
    • or API key in cookie cave_access
    • or API key in cookie __Secure-cave_identity.session_token

Request body

namestring required
invariantstring required
grader_typestring required
subject_columnstring required
label_columnstring required
identity_columnstring required
challenge_version_idstring uuid required
population_version_idstring uuid required
sampling_method'random' | 'systematic' | 'complete_population' required
population_definitionstring required
label_provenancestring required
agent_idstring uuid nullable
workflow_idstring uuid nullable

Response

Immutable independent set registered. Labels and subject semantics are supplied by authorized native users, never the evaluator author.

Changes

No recorded changes to this operation across all 1 revision of this API.