ABExperimentService5

Account1

Audio5

Batches4

Billing1

Chat1

Code Interpreter2

Completion1

ConfigService2

Deployments6

DeploymentService10

DeploymentsStorage2

DeploymentsVolumes2

Embeddings1

endpoints1

Endpoints8

EndpointService9

evaluation5

Files5

Fine-tuning14

GPUClusterService5

Hardware1

HealthCheckService1

Images1

InstanceClusterAddOnService5

InstanceTypeService2

Jobs2

Models2

PlacementProfileService2

Queue5

RegionService1

RemediationService6

Rerank1

RL29

RolloutService10

Secrets5

ShadowExperimentService10

SharedVolumeService5

SupportedModelsService2

Video2

Voices1

Volumes3

VolumeService13

evaluation

Create an evaluation job

post/evaluation

Request body

type'classify' | 'score' | 'compare' required

The type of evaluation to perform

Example request

{
  "type": "classify",
  "parameters": {
    "judge": {
      "model": "Qwen/Qwen3.5-9B",
      "system_template": "Imagine you are a helpful assistant",
      "num_workers": 5,
      "max_tokens": 8192
    },
    "labels": [
      "yes",
      "no"
    ],
    "pass_labels": [
      "yes"
    ],
    "model_to_evaluate": {
      "model": "Qwen/Qwen3.5-9B",
      "max_tokens": 512,
      "temperature": 0.7,
      "system_template": "You are a helpful assistant.",
      "input_template": "Please answer the following question: {{ question }}",
      "num_workers": 5
    },
    "input_data_file_path": "file-1234-aefd"
  }
}

Response

Evaluation job created successfully

workflow_idstring

The ID of the created evaluation job

status'pending'

Initial status of the job

Example response

{
  "workflow_id": "eval-1234-1244513"
}

Changes

Changed in 1 of the 14 revisions of this API.10

Jul 26, 20260c4fdad7743510See the full diff
- ○
  the parameters/oneOf[EvaluationClassifyParameters]/judge/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationClassifyParameters]/model_to_evaluate/oneOf[EvaluationModelRequest]/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationCompareParameters]/judge/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationCompareParameters]/model_a/oneOf[EvaluationModelRequest]/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationCompareParameters]/model_b/oneOf[EvaluationModelRequest]/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationScoreParameters]/judge/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationScoreParameters]/max_score request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationScoreParameters]/min_score request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationScoreParameters]/model_to_evaluate/oneOf[EvaluationModelRequest]/temperature request property type/format was generalized from number/float to number/
  request-property-type-generalized
- ○
  the parameters/oneOf[EvaluationScoreParameters]/pass_threshold request property type/format was generalized from number/float to number/
  request-property-type-generalized
This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

Full history