scorer-feedback

Generate Prompt

Changed on

Generate improved scorer prompt using Autotune (CLHF).

This endpoint:

  1. Fetches all feedback items from the active queue with ClickHouse context
  2. Assembles the current metric system prompt
  3. Constructs AutotuneInput with annotated examples
  4. Queues a background job to the runners service
  5. Returns task_id for polling

The API prepares all data internally - no request body needed.

Validation:

  • Scorer exists and user has access
  • Active queue exists in 'pending' state
  • Queue has at least one feedback item

Response:

  • 202 Accepted with task_id for polling

Errors:

  • 404 - Scorer not found or no active pending queue exists
  • 409 - Queue is already locked (generating or reviewing)
  • 422 - Scorer missing required configuration or queue has no feedback items
post/feedback-queues/scorer/{scorer_id}/generate-prompt

Request

  • Base URL
    https://api.galileo.ai
  • URL
    https://api.galileo.ai/feedback-queues/scorer/{scorer_id}/generate-prompt
  • Auth
    • API key in header Splunk-AO-API-Key
    • or API key in header Galileo-API-Key
    • or OAuth 2
      Password
      Token URL: https://api.galileo.ai/login
    • or HTTP basic

Path parameters

scorer_idstring uuid4 required

Response

Successful Response

task_idstring required

Celery task ID for polling

queue_idstring uuid4 required

Queue ID being processed

scorer_idstring uuid4 required

Scorer ID

feedback_countinteger required

Number of feedback items

statusstring required

Current status (generating)

messagestring required

Message with polling instructions

Changes