CAcaveman

POST /projects/{projectId}/eval-datasets

POST /api/v1/projects/{projectId}/eval-datasets on the Caveman Cloud control plane. Requires the eval:write scope. Exposed to autonomous agents as the create_dataset action of the caveman_evals_write tool (write access). Author versioned datasets, reusable evaluators and workbenches; create strict suites, ingest evidence, and run baselines. Evidence ingest supports dry_run — always dry-run first. Ask Caveman to draft quality criteria again within the drafting allowance, propose a criterion for a person to turn on, and hold the pull request that adds the Caveman check to CI. Track a criterion on a slice of traces in one step: from an agent it lands proposed, and a person turns it on. Queue units for review and label them (an agent's label is recorded as an agent's, never as human gold), and make a dataset from recorded traffic.

post/projects/{projectId}/eval-datasets

Request

  • Base URL
    https://api.caveman.so/api/v1
  • URL
    https://api.caveman.so/api/v1/projects/{projectId}/eval-datasets
  • Auth
    • OAuth 2
      Authorization code: /api/v1/oauth/authorize
      Token URL: /api/v1/oauth/token scopes: eval:write
    • or HTTP bearer
    • or API key in cookie cave_access
    • or API key in cookie __Secure-cave_identity.session_token

Request body

namestring required
descriptionstring
upsertboolean

Update the live resource of this name in the project instead of refusing the name. A dataset whose rows are unchanged is left as it is, so its next run reuses its version.

Response

Successful response. Exact status depends on synchronous or durable delivery outcome.

Changes

No recorded changes to this operation across all 1 revision of this API.