--- title: "PUT /projects/{projectId}/eval-workbenches/{workbenchId}" method: PUT path: "/projects/{projectId}/eval-workbenches/{workbenchId}" --- # PUT /projects/{projectId}/eval-workbenches/{workbenchId} `PUT /projects/{projectId}/eval-workbenches/{workbenchId}` ## Request - Base URL: `https://api.caveman.so/api/v1` - URL: `https://api.caveman.so/api/v1/projects/{projectId}/eval-workbenches/{workbenchId}` - Auth: one of: - OAuth 2 - Authorization code: `/api/v1/oauth/authorize` - Token URL: `/api/v1/oauth/token` scopes: `eval:write` - HTTP bearer - API key in cookie `cave_access` - API key in cookie `__Secure-cave_identity.session_token` PUT /api/v1/projects/{projectId}/eval-workbenches/{workbenchId} on the Caveman Cloud control plane. Requires the eval:write scope. Exposed to autonomous agents as the update_workbench action of the caveman_evals_write tool (write access). Author versioned datasets, reusable evaluators and workbenches; create strict suites, ingest evidence, and run baselines. Evidence ingest supports dry_run — always dry-run first. Ask Caveman to draft quality criteria again within the drafting allowance, propose a criterion for a person to turn on, and hold the pull request that adds the Caveman check to CI. Track a criterion on a slice of traces in one step: from an agent it lands proposed, and a person turns it on. Queue units for review and label them (an agent's label is recorded as an agent's, never as human gold), and make a dataset from recorded traffic. ## Request body - object - `revision` integer, required - `name` string, required - `description` string - `document` object, required: Versioned dataset-target-evaluator document used by UI, REST, CI and agent tools. - `schema_version` unknown, required - `datasets` object[], required - `key` string, required - `dataset_id` string, uuid, required - `version` integer - `document_sha256` string - `active_dataset_key` string - `targets` union[], required - union - object - `key` string, required - `name` string, required - `kind` 'prompt', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `prompt_id` string, uuid, required - `prompt_version` integer, required - `provider` string, required - `model` string, required - `temperature` number - `max_tokens` integer - `response_format` 'text' | 'json' - object - `key` string, required - `name` string, required - `kind` 'agent', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `agent_id` string, uuid, required - `workflow_id` string, uuid - `endpoint` string, uri, required - `method` 'POST', required - `authentication` object, required - … - `output_path` string - object - `key` string, required - `name` string, required - `kind` 'workflow', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `workflow_id` string, uuid, required - `endpoint` string, uri, required - `method` 'POST', required - `authentication` object, required - … - `output_path` string - object - `key` string, required - `name` string, required - `kind` 'recorded', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `output_column` string, required - object - `key` string, required - `name` string, required - `kind` 'external', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required: The caller ran this step: a run of it carries each output (evals.run outputs). At most one per document. - object - `key` string, required - `name` string, required - `kind` 'http', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `endpoint` string, uri, required - `method` 'POST', required - `authentication` object, required - … - `output_path` string - object - `key` string, required - `name` string, required - `kind` 'model', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `provider` string, required - `model` string, required - `temperature` number - `max_tokens` integer - `response_format` 'text' | 'json' - `user_template` string, required - `system_template` string - object - `key` string, required - `name` string, required - `kind` 'evaluator', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `evaluator_id` string, uuid, required - `evaluator_version` integer, required - `config_sha256` string, required - object - `key` string, required - `name` string, required - `kind` 'scenario', required - `input_mapping` object, required - `dataset_input_mappings` object - `config` object, required - `dataset_id` string, uuid, required - `version` integer, required - `scenario_sha256` string, required - `repository_commit_sha` string, required - `recipe_id` string, required - `recipe_sha256` string, required - `evaluators` union[], required - union - object - `key` string, required - `name` string, required - `source` 'saved', required - `target_keys` string[] - `evaluator_id` string, uuid, required - `evaluator_version` integer, required - `config_sha256` string, required - `input_mapping` object - `candidate` object, required - … - `reference` object - … - `baseline` object - … - `dataset_input_mappings` object - object - `key` string, required - `name` string, required - `source` 'inline', required - `target_keys` string[] - `grader_type` string, required - `options` object, required - `input_mapping` object, required - `candidate` object, required - … - `reference` object - … - `baseline` object - … - `dataset_input_mappings` object - object - `key` string, required - `name` string, required - `source` 'criterion', required - `target_keys` string[] - `criterion_id` string, uuid, required - `criterion_version` integer, required - `input_mapping` object - `candidate` object, required - … - `reference` object - … - `baseline` object - … - `dataset_input_mappings` object - `settings` object - `concurrency` integer, required - `row_height` 'compact' | 'comfortable', required - `trials` integer - `judge_order` 'randomized' | 'fixed' - `judge_seed` integer - `comparison` object - `primary_evaluator_key` string - `confidence_level` unknown - `minimum_samples` integer - `baseline_target_key` string ## Response `2XX` Successful response. Exact status depends on synchronous or durable delivery outcome. ## Other responses - `4XX`: Structured failure. `error.code` is stable; `error.request_id` is echoed in the x-cave-request-id header. - `5XX`: Structured failure. `error.code` is stable; `error.request_id` is echoed in the x-cave-request-id header. --- [API](https://skmtc.dev/caveman/apis/caveman-cloud-api.md) - [All operations](https://skmtc.dev/caveman/apis/caveman-cloud-api/llms.txt) - [OpenAPI document](https://skmtc.dev/caveman/apis/caveman-cloud-api/revisions/508312d01cfc?raw)