scorer-feedback

Validate Autotune Queue

Changed on

Test a draft prompt against feedback records in the active queue.

Creates a metrics testing run by copying the feedback records and running generated scorer validation with the provided prompt. Used in the AutoTune Data tab for 3-column comparison: Original | Expected (Annotated) | New Result.

Works in any active queue state (pending, generating, or reviewing).

Response:

  • 202 Accepted with metrics_testing_run_id for viewing results

Errors:

  • 404 - Scorer not found or no active queue exists
  • 422 - Queue has no feedback items or scorer has no version
  • 500 - Record processing failure
post/feedback-queues/scorer/{scorer_id}/validation-runs

Request

  • Base URL
    https://api.galileo.ai
  • URL
    https://api.galileo.ai/feedback-queues/scorer/{scorer_id}/validation-runs
  • Auth
    • API key in header Splunk-AO-API-Key
    • or API key in header Galileo-API-Key
    • or OAuth 2
      Password
      Token URL: https://api.galileo.ai/login
    • or HTTP basic

Path parameters

scorer_idstring uuid4 required

Request body

user_promptstring required

The draft/edited prompt to test against feedback records

Response

Successful Response

metrics_testing_run_idstring uuid4 required

ID of the metrics testing run containing validation results

project_idstring uuid4 required

Project where the metrics testing run was created

queue_idstring uuid4 required

Queue ID that was tested

feedback_countinteger required

Number of feedback records tested

Changes