experiment

Create Experiment

Create a new experiment for a project.

post/public/v2/projects/{project_id}/experiments

Path parameters

project_idstring uuid4 required

Request body

namestring required
playground_idstring uuid4 nullable
prompt_template_version_idstring uuid4 nullable
playground_prompt_idstring uuid4 nullable
triggerboolean
experiment_group_idstring uuid4 nullable
experiment_group_namestring nullable

Example request

{
  "scorers": [
    {
      "filters": [
        {
          "value": "ENUM_VALUE"
        }
      ]
    }
  ]
}

Response

Successful Response

idstring uuid4 required

Galileo ID of the experiment

created_atstring date-time

Timestamp of the experiment's creation

updated_atstring date-time nullable

Timestamp of the trace or span's last update

namestring

Name of the experiment

project_idstring uuid4 required

Galileo ID of the project associated with this experiment

created_bystring uuid4 nullable
num_spansinteger nullable
num_tracesinteger nullable
num_sessionsinteger nullable
task_type13 | 15 | 16 | 17 | 18 required

Valid task types for modeling.

We store these as ints instead of strings because we will be looking this up in the database frequently.

aggregate_metricsobject
structured_aggregate_metricsobject nullable

Structured aggregate metrics with full statistical aggregates (avg, min, max, sum, count). Keys are scorer UUIDs for scorer-backed metrics (matching available_columns column IDs after stripping the 'metrics/' prefix) and raw strings for system metrics (e.g. 'duration_ns', 'cost'). Present only when use_clickhouse_run_aggregates flag is enabled.

aggregate_feedbackobject

Aggregate feedback information related to the experiment (traces only)

rating_aggregatesobject

Annotation aggregates keyed by template ID and root type

ranking_scorenumber nullable
rankinteger nullable
winnerboolean nullable
playground_idstring uuid4 nullable
prompt_modelstring nullable
tagsobject
experiment_group_idstring uuid4 nullable
experiment_group_namestring nullable
experiment_group_is_systemboolean nullable

Changes

Changed in 5 of the 14 revisions of this API.10456

    • added the new optional request property scorers/items/chainpoll_template

      new-optional-request-property

    • added the new optional request property scorers/items/class_name_to_vocab_ix

      new-optional-request-property

    • added the new optional request property scorers/items/default_model_alias

      new-optional-request-property

    • added the new optional request property scorers/items/ground_truth

      new-optional-request-property

    • added the new optional request property scorers/items/lora_task_id

      new-optional-request-property

    • added the new optional request property scorers/items/lora_weights_path

      new-optional-request-property

    • added the new optional request property scorers/items/luna_input_type

      new-optional-request-property

    • added the new optional request property scorers/items/luna_output_type

      new-optional-request-property

    • added the new optional request property scorers/items/metric_name

      new-optional-request-property

    • added the new optional request property scorers/items/model_alias

      new-optional-request-property

    • added the new optional request property scorers/items/model_type

      new-optional-request-property

    • added the new optional request property scorers/items/prompt

      new-optional-request-property

    • added the new optional request property scorers/items/registered_scorer_id

      new-optional-request-property

    • added the new optional request property scorers/items/required_metric_ids

      new-optional-request-property

    • added the new optional request property scorers/items/required_scorers

      new-optional-request-property

    • added the new optional request property scorers/items/requires_tools_in_llm_span

      new-optional-request-property

    • added the new optional request property scorers/items/roll_up_strategy

      new-optional-request-property

    • added the new optional request property scorers/items/scorer_path_name

      new-optional-request-property

    • added the new optional request property scorers/items/user_code

      new-optional-request-property

    This revision also has 15 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • 344acdc36dbb11See the full diff
    • removed galileo_core__schemas__shared__scorers__scorer_name__ScorerName from the scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor request property anyOf list

      request-property-any-of-removed

    • added ScorerName to the scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor request property anyOf list

      request-property-any-of-added

    This revision also has 44 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

    • added the new optional request property prompt_settings/anyOf[subschema #1: PromptRunSettings]/known_models/items/deprecation_date

      new-optional-request-property

    • added the new optional request property prompt_settings/anyOf[subschema #1: PromptRunSettings]/known_models/items/lifecycle_state

      new-optional-request-property

    • added the new optional request property prompt_settings/anyOf[subschema #1: PromptRunSettings]/known_models/items/replacement_alias

      new-optional-request-property

    • added the new optional request property prompt_settings/anyOf[subschema #1: PromptRunSettings]/known_models/items/retirement_date

      new-optional-request-property

    • added the new agent_flow_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new agent_flow_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new conversation_quality_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new conversation_quality_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new input_sexist_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new input_sexist_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new instruction_adherence_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new instruction_adherence_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new output_sexist_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new output_sexist_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new prompt_injection_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new prompt_injection_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new tool_selection_quality_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new tool_selection_quality_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • removed the 12.00 enum value from the task_type response property for the response status 200

      response-property-enum-value-removed

    • removed the 7.00 enum value from the task_type response property for the response status 200

      response-property-enum-value-removed

    • removed the 9.00 enum value from the task_type response property for the response status 200

      response-property-enum-value-removed

    This revision also has 8 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • 1f7a6a4a81ad544See the full diff
    • removed the enum value code of the request property scorers/items/scorer_type

      request-property-enum-value-removed

    • removed the enum value llm of the request property scorers/items/scorer_type

      request-property-enum-value-removed

    • removed the enum value luna of the request property scorers/items/scorer_type

      request-property-enum-value-removed

    • removed the enum value preset of the request property scorers/items/scorer_type

      request-property-enum-value-removed

    • the scorers/items/scorer_type request property type/format changed from string/ to /

      request-property-type-changed

    • removed the request property scorers/items/model_type

      request-property-removed

    • removed the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/input_type

      request-property-removed

    • removed the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/output_type

      request-property-removed

    • removed the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/scoreable_node_types

      request-property-removed

    • added the new optional request property scorers/items/roll_up_config

      new-optional-request-property

    • added the new optional request property scorers/items/scorer_version_id

      new-optional-request-property

    • added ScorerTypes subschema #2 to the scorers/items/scorer_type request property anyOf list

      request-property-any-of-added

    • the request property scorers/items/scorer_type became optional

      request-property-became-optional

    This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • 05003e7835a2411See the full diff
    • removed the enum value bleu of the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-removed

    • removed the enum value prompt_perplexity of the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-removed

    • removed the enum value rouge of the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-removed

    • removed the enum value uncertainty of the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-removed

    • the endpoint scheme security ClassicAPIKeyHeader was added to the API

      api-security-added

    • added the new action_completion_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new action_completion_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new correctness_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new correctness_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new ground_truth_adherence_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new ground_truth_adherence_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new reasoning_coherence_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new reasoning_coherence_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new user_intent_change_audio enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    • added the new user_intent_change_vision enum value to the request property scorers/items/scorer_version/anyOf[subschema #1: BaseScorerVersionDB]/finetuned_scorer/anyOf[subschema #1: BaseFinetunedScorerDB]/executor/anyOf[subschema #1: ScorerName]/

      request-property-enum-value-added

    This revision also has 15 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog