Tasks

Create task

Changed on

Creates a new task. Supported task types:

typeData sourceNotes
TEMPLATE_EVALUATIONproject_id or dataset_idRequires evaluators. Supports continuous operation.
CODE_EVALUATIONproject_id or dataset_idRequires evaluators. Supports continuous operation.
RUN_EXPERIMENTdataset_id onlyRequires run_configuration. Never continuous.

For RUN_EXPERIMENT tasks the run configuration is stored on the task. Each trigger (POST /v2/tasks/{task_id}/trigger) supplies per-run fields (experiment_name, optional example subset, etc.) and starts an async run. Poll GET /v2/task-runs/{run_id} until status reaches a terminal state.

Payload Requirements (template_evaluation / code_evaluation)

  • At least one evaluator is required.
  • Duplicate evaluator IDs are not allowed.
  • When dataset_id is provided, experiment_ids must contain at least one entry.
  • sampling_rate and is_continuous are only supported on project-based tasks.
  • System-managed fields (id, created_at, updated_at) are rejected on input.
  • evaluator_version_id pins an evaluator to one version. Omit it (or send null) to run that evaluator's latest version, which is the default. The version must belong to the evaluator named by evaluator_id, and every evaluator on the task must resolve to the same data scope — both return 422. List an evaluator's versions with GET /v2/evaluators/{evaluator_id}/versions.

Payload Requirements (run_experiment)

  • dataset_id is required; project_id must be omitted.
  • run_configuration is required; evaluators, experiment_ids, sampling_rate, is_continuous, and query_filter must be omitted.

Valid example (template_evaluation, project-based)

{
  "name": "Production Hallucination Check",
  "type": "TEMPLATE_EVALUATION",
  "project_id": "TW9kZWw6MTIzOmFCY0Q=",
  "sampling_rate": 1.0,
  "is_continuous": true,
  "evaluators": [
    {
      "evaluator_id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
      "column_mappings": {"input": "attributes.input.value", "output": "attributes.output.value"}
    }
  ]
}

Valid example (pinned to a specific evaluator version)

{
  "name": "Hallucination Check v3",
  "type": "TEMPLATE_EVALUATION",
  "project_id": "TW9kZWw6MTIzOmFCY0Q=",
  "evaluators": [
    {
      "evaluator_id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
      "evaluator_version_id": "RXZhbHVhdG9yVmVyc2lvbjo5OTphQmNE",
      "column_mappings": {"input": "attributes.input.value"}
    }
  ]
}

Invalid example (run_experiment missing run_configuration)

{
  "name": "My Experiment",
  "type": "RUN_EXPERIMENT",
  "dataset_id": "RGF0YXNldDo1NjpxUndY"
}

Invalid example (422 — the version belongs to a different evaluator)

{
  "name": "Mismatched Pin",
  "type": "TEMPLATE_EVALUATION",
  "project_id": "TW9kZWw6MTIzOmFCY0Q=",
  "evaluators": [
    {
      "evaluator_id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
      "evaluator_version_id": "RXZhbHVhdG9yVmVyc2lvbjo3OmFCY0Q="
    }
  ]
}

<Note>This endpoint is in beta, read more here.</Note>

post/v2/tasks

Request

  • Base URL
    https://api.arize.com
  • URL
    https://api.arize.com/v2/tasks
  • Auth
    HTTP bearer

Request body

OR
OR

Response

Returns a single task object

idstring required

The unique identifier for the task

namestring required

The name of the task

type'TEMPLATE_EVALUATION' | 'CODE_EVALUATION' | 'RUN_EXPERIMENT' required

The task type.

  • TEMPLATE_EVALUATION - An LLM template-based evaluation task.
  • CODE_EVALUATION - A code-based evaluation task.
  • RUN_EXPERIMENT - A task that runs experiments.
project_idstring nullable

The project identifier (base64). Present for project-based tasks.

dataset_idstring nullable

The dataset identifier (base64). Present for dataset-based tasks.

sampling_ratenumber nullable

Sampling rate between 0 and 1. Only applicable for project-based tasks.

is_continuousboolean required

Whether the task runs continuously on incoming data.

query_filterstring nullable required

Task-level query filter applied to all data. Span-granularity shape only. Null when the task uses the trace/session shape (query_filters). Mutually exclusive with query_filters.

experiment_idsstring[] required

Experiment identifiers (base64) for dataset-based tasks.

last_run_atstring date-time nullable required

When the task was last run.

created_atstring date-time required

When the task was created.

updated_atstring date-time required

When the task was last updated.

created_by_user_idstring nullable required

The unique identifier for the user who created the task.

Changes

  • c972f4a6595cRevision changes
    • warning

      the //////////// request property's maxLength was set to 1024

    • warning

      the //////////// request property's maxLength was set to 256

    • warning

      the ///////// request property's maxLength was set to 512

    • warning

      the /////////// request property's maxLength was set to 2000

    • warning

      the //////////// request property's maxLength was set to 1024

    • warning

      the //////////// request property's maxLength was set to 256

    • warning

      the ///////// request property's maxLength was set to 512

    • warning

      the /////////// request property's maxLength was set to 2000

  • f63ac334fd4aRevision changes
    • breaking

      the ///// request property type changed from object to no type

    • breaking

      the ///// request property type changed from object to no type

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • warning

      removed the request property //////

    • info

      added the new optional request property ////

    • info

      added the new optional request property ////

    • info

      added to the ///// request property oneOf list

    • info

      added to the ///// request property oneOf list

    • info

      added the optional property // to the response with the status

    • info

      added the optional property to the response with the status