evaluators

Submit Evaluator Judgment

Submit Evaluator judgment for an existing Log.

Creates a new Log. The evaluated Log will be set as the parent of the created Log.

post/evaluators/log

Request

  • Base URL: https://api.humanloop.com/v5
  • URL: https://api.humanloop.com/v5/evaluators/log
  • Auth: API key in header X-API-KEY

Query parameters

version_idstring

ID of the Evaluator version to log against.

environmentstring

Name of the Environment identifying a deployed version to log to.

Request body

pathstring

Path of the Evaluator, including the name. This locates the Evaluator in the Humanloop filesystem and is used as as a unique identifier. For example: folder/name or just name.

idstring

ID for an existing Evaluator.

start_timestring date-time

When the logged event started.

end_timestring date-time

When the logged event ended.

outputstring

Generated output from the LLM. Only populated for LLM Evaluator Logs.

created_atstring date-time

User defined timestamp for when the log was created.

errorstring

Error message if the log is an error.

provider_latencynumber double

Duration of the logged event in seconds.

stdoutstring

Captured log and debug statements.

provider_requestobject

Raw request sent to provider. Only populated for LLM Evaluator Logs.

provider_responseobject

Raw response received the provider. Only populated for LLM Evaluator Logs.

inputsobject

The inputs passed to the prompt template.

sourcestring

Identifies where the model was called from.

metadataobject

Any additional metadata to record.

parent_idstring required

Identifier of the evaluated Log. The newly created Log will have this one set as parent.

source_datapoint_idstring

Unique identifier for the Datapoint that this Log is derived from. This can be used by Humanloop to associate Logs to Evaluations. If provided, Humanloop will automatically associate this Log to Evaluations that require a Log for this Datapoint-Version pair.

trace_parent_idstring

The ID of the parent Log to nest this Log under in a Trace.

userstring

End-user ID related to the Log.

environmentstring

The name of the Environment the Log is associated to.

saveboolean

Whether the request/response payloads will be stored on Humanloop.

log_idstring

This will identify a Log. If you don't provide a Log ID, Humanloop will generate one for you.

marked_completedboolean

Whether the Log has been manually marked as completed by a user.

Response

Successful Response

idstring required

String identifier of the new Log.

parent_idstring required

Identifier of the evaluated parent Log.

session_idstring

Identifier of the Session containing both the parent and the new child Log. If the parent Log does not belong to a Session, a new Session is created with this ID.

version_idstring required

Identifier of Evaluator Version for which the Log was registered.

Changes

No recorded changes to this operation across all 1 revision of this API.