autoresearch
autoresearch

Run online validation

Validate predictions against realized outcomes for all matured prediction dates. A prediction date is matured when today >= prediction_date + horizon_days. Computes realized AUC, Brier score, calibration error (ECE), and lift@10/20 per model. Updates the model's realized_score, calibration_error, and clears the is_preliminary flag. Already-validated dates are skipped. In production this is triggered by the daily Temporal validation workflow after inference runs.

post/api/projects/{project_id}/autoresearch/{id}/validate_online/

Request

  • Base URL: https://us.posthog.com
  • URL: https://us.posthog.com/api/projects/{project_id}/autoresearch/{id}/validate_online/
  • Auth: HTTP bearer

Path parameters

idstring uuid required

A UUID string identifying this autoresearch pipeline.

project_idstring required

Project ID of the project you're trying to access. To find the ID of the project, make a call to /api/projects/.

Response

One AutoresearchRun per matured prediction date that was validated. Empty list when no prediction dates have matured yet.

idstring uuid required

Unique UUID of this run.

pipelinestring uuid required

Pipeline this run belongs to.

modelstring uuid nullable

Model used for scoring. Null for validation runs.

run_type'inference' | 'validation' required
  • inference - Inference
  • validation - Validation
status'pending' | 'running' | 'completed' | 'failed'
  • pending - Pending
  • running - Running
  • completed - Completed
  • failed - Failed
rows_scoredinteger nullable

Number of users scored in this inference run.

metricsobject required

Run metrics: score distribution summary, validation AUC, etc. An inference run records 'rows_eligible', the users in the inference population. When it is larger than rows_scored, the run scored a rolling part of the population: users never scored first, then users whose last score was oldest.

errorstring

Error message if the run failed.

started_atstring date-time nullable

Timestamp when the run started.

completed_atstring date-time nullable

Timestamp when the run completed or failed.

created_atstring date-time required

Changes

No recorded changes to this operation across all 1 revision of this API.