Sign in
Factual Consistency Evaluation

Evaluate factual consistency

Evaluates the factual consistency of a generated text (like a summary) against source documents. The evaluation determines how accurately the generated text reflects the information in the source documents. This helps identify potential hallucinations or misrepresentations.

Use this endpoint to validate generated content against trusted source materials, such as in legal, healthcare, scientific publishing, and enterprise knowledge systems.

The request body includes the following parameters:

  • model_parameters.model_name: Optional. The evaluation model to use. hhem_v2.3 is the default and the recommended model. hhem_v2.2 is retired; it remains accepted for backward compatibility and is served by hhem_v2.3. Any other value is rejected with a 400.
  • generated_text: The output text you want to evaluate, such as a model-generated summary, answer, or response.
  • source_texts: An array of source documents or passages used to verify the accuracy of the generated text.

The endpoint scores the texts as given, in whatever language they are written. HHEM is trained on eng, deu, fra, spa, por, ara, kor, zho, rus, jpn, and hin; treat scores for text in other languages as unreliable.

Example request

This example evaluates whether a generated statement about the Eiffel Tower is factually accurate based on two reference documents.

{
  "generated_text": "The Eiffel Tower is located in Berlin.",
  "source_texts": [
    "The Eiffel Tower is a famous landmark located in Paris, France.",
    "It was built in 1889 and remains one of the most visited monuments in the world."
  ]
}

Example response

The response includes the factual consistency score.

{
  "score": 0.23
}
  • score: A normalized value between 0.0 and 1.0 that reflects the overall factual alignment between the generated text and the source texts. Higher scores indicate stronger consistency.
post/v2/evaluate_factual_consistency

Headers

Request-Timeoutinteger

The platform makes a best effort to complete the request in the specified seconds, or it times out.

Request-Timeout-Millisinteger

The platform makes a best effort to complete the request in the specified milliseconds, or it times out.

Request body

generated_textstring required

The generated text (e.g., summary or answer) to evaluate for factual consistency.

source_textsstring[] required

The source documents or text snippets against which to evaluate factual consistency.

Example request

{
  "model_parameters": {
    "model_name": "hhem_v2.3"
  }
}

Response

The factual consistency evaluation results.

scorenumber double required

A prediction score from HHEM, greater than 0 and less than 1. The score monotonically indicates the likelihood of hallucinations.

Changes

Changed in 1 of the 8 revisions of this API.1