Safety

Detect KOSA-aligned harm signals in text

Changed on

Comprehensive safety analysis covering all KOSA harm categories: self-harm and suicidal ideation, eating disorders, substance use, depression/anxiety indicators, compulsive usage patterns, sexual content, violence, and hate speech. Returns categorized findings with age-calibrated severity scoring. This is the recommended STARTING endpoint for broad content moderation, not a complete safety layer on its own. Pair it with /safety/bullying, or call /api/v1/analyse/multi. Peer-to-peer harassment, exposure or "fisha" accounts, and image-based abuse of a named person are scored by /safety/bullying; grooming trajectories by /safety/grooming; extremist recruitment by /safety/radicalisation. This endpoint also reports unsafe: false where the sender is DESCRIBING a harm rather than being subject to it (survivor testimony, news, a lesson, a child relaying something seen elsewhere), which is deliberate and keeps protective speech out of your queue, but means a message another detector would flag can come back clear here. Measured on our published kids-online-slang set, every case in that shape was caught by a sibling detector. If you call only this endpoint you inherit that boundary.

post/api/v1/safety/unsafe

Request

  • Base URL: https://api.tuteliq.ai
  • URL: https://api.tuteliq.ai/api/v1/safety/unsafe
  • Auth: one of:
    • HTTP bearer
    • API key in header x-api-key

Request body

textstring required

The content to analyze

support_threshold'low' | 'medium' | 'high' | 'critical'

Top-level alias for options.support_threshold. Minimum severity to show crisis support resources (default: high). Critical always shows.

continuation_tokenstring

Opaque signed token from a previous /unsafe call. Carries derived analysis state — no message content is stored server-side.

reset_conversationboolean

Discard any continuation_token and start a fresh conversation.

new_conversationboolean

Cross into a new conversation WITHOUT discarding the sustained_hopelessness pattern carried in continuation_token — unlike reset_conversation, which discards everything. Requires continuation_token to also be set. Use this when a new conversation starts on your side (e.g. a new chat session days later) but you still want Tuteliq to notice if the same pattern keeps recurring across those sessions.

message_idstring

Optional customer-side message id for evidence pointers.

external_idstring

Your unique identifier (e.g. message ID, content ID) for correlating Tuteliq results with your own system. Echoed back in the response and included in webhook payloads so you can match alerts to the original content.

customer_idstring

Your end-customer identifier for multi-tenant / B2B2C scenarios. If you serve multiple customers (schools, apps, brands) from a single API key, set this to route webhook alerts to the correct customer. Echoed back in the response and included in webhook payloads as "customerId".

metadataobject

Arbitrary key-value pairs for additional context (e.g. { "channel": "discord", "region": "eu" }). Stored with the detection result, echoed in the response, and included in webhook payloads. Maximum 20 properties.

incident_moderation_enabledboolean

Per-call override of your account incident logging setting. When set, it takes precedence over the account-level flag for THIS request: true forces the incident to be persisted, false suppresses persistence. Omit to use your account default (which itself defaults to enabled). Lets you control incident logging per request, e.g. suppress logging for test traffic.

Response

Default Response

urgency_keptboolean

True when a declared context.conversation_type would have capped an urgent action to flag_for_review and did not, because the case is acute: the analysis named an acute marker, or a separate reading found a person at risk now with something happening as the text is written. A reading that gave no answer also keeps the urgency. Absent otherwise, and never present together with conversation_type_applied.

urgency_kept_reasonstring

Human-readable explanation. Present only when urgency_kept is true.

conversation_type_appliedstring

Present only when a caller-declared context.conversation_type actually changed routing. Confirms the setting was taken into account, rather than leaving the platform to assume it was.

unsafeboolean

Unsafe-content signal observed at ANY severity (including low / monitor cases). For production branching prefer recommended_action (flag_for_review / block / immediate_intervention), which is a stable enum, over the bare presence of a signal. SCOPED TO THIS ENDPOINT: false means no KOSA-category signal was found HERE, not that the content is safe. It is also false where the sender is describing a harm rather than being subject to it. Peer harassment and image-based abuse of a named person are scored by /safety/bullying, so do not read false as a clearance without also calling that endpoint or /api/v1/analyse/multi.

categoriesstring[]
severity'low' | 'medium' | 'high' | 'critical'
confidencenumber
risk_scorenumber
rationalestring
recommended_action'none' | 'monitor' | 'flag_for_review' | 'block' | 'immediate_intervention'

Routing key, weakest to strongest: none, monitor, flag_for_review, block, immediate_intervention. Safe to switch on.

action_detailstring

Optional human-readable expansion of recommended_action for a moderator UI. Free text: do not branch on it.

context_factorsstring[]

Named influences on the severity assessment, restricted to values that are safe to publish. Currently only stated_test_frame, set when the speaker themselves called this a test, demo or rehearsal. DESCRIPTIVE: the statement is still scored as if genuine, because presenting a disclosure as a test is also how a real one gets minimised, so this marks the frame for a reviewer rather than discounting it. Absent when nothing applies. ADVISORY: for a moderator UI and triage queues. Do not branch production logic on it; use recommended_action, which is a stable enum.

languagestring

Language code used for analysis (auto-detected or provided)

language_status'stable' | 'beta'

Language support maturity: "stable" (en) or "beta" (all others)

credits_usednumber

Number of credits consumed by this request

continuation_tokenstring

Signed token carrying derived analysis state for the next call. Server stores nothing.

continuation_expires_atstring

ISO 8601 expiry timestamp of the continuation_token.

state_source'token' | 'fresh' | 'reset'

How prior state was sourced.

trajectory_risknumber

Conversation-level risk 0-1, distinct from risk_score which scores only this message. Anchored on the highest severity seen so far and decaying slowly across benign turns. Absent on the first turn. Derived from the signed token: no message content is stored to produce it.

trajectory'rising' | 'stable' | 'declining' | 'none'

Direction of travel across the conversation so far.

severity_seriesnumber[]

Per-turn severity, oldest first — the evidence behind trajectory_risk.

sustained_hopelessness_conversationsnumber

Count of distinct conversations, within the last 30 days, that named the hopelessness_sustained marker at least once. Only present when new_conversation has been used to cross at least one conversation boundary. recommended_action is floored to at least flag_for_review once this reaches 3.

escalation_cappedboolean

True when a critical-severity coded-term match corroborated this detection but the model's own independent read wasn't itself concerned enough to also corroborate escalation to immediate_intervention. Absent when no cap applied.

escalation_capped_reasonstring

Human-readable explanation of the cap. Present only when escalation_capped is true.

third_party_concernboolean

True when this detection rests on a child reporting, disclosing or reacting to a real risk involving somebody, rather than on anyone in the conversation enacting one -- a peer saying another child has self-harm marks, that a sibling is using again, that a suicide challenge is going round. Route these to a welfare/safeguarding queue rather than an intervention queue: the speaker is the person who spoke up, and the risk may belong to somebody who is not in the conversation. Set only when the verdict rests on that report: if the analysis independently called the content unsafe, this is absent and recommended_action carries the answer. This path itself raises neither severity nor action, but it is NOT a cap on the response -- always route on recommended_action, which can be anything up to immediate_intervention. Absent unless it applied.

third_party_concern_reasonstring

Human-readable explanation. Present only when third_party_concern is true.

declared_age_overriddenboolean

True when the caller declared an adult age group but the speaker states a minor age in the message or in the earlier turns sent with it. The declared age was not used. Nothing is escalated by this on its own; the analysis runs as if no age had been sent. Worth investigating at your end: an age check that disagrees with what the user says is what a bypassed age verification looks like. A stated ADULT age never has this effect. Absent unless it applied.

declared_age_overridden_reasonstring

Human-readable explanation. Present only when declared_age_overridden is true.

protective_speech_cappedboolean

True when an alert was lowered because a separate reading found a text working against the harm it describes: teaching, moderating, counselling, reporting or asking for advice. Two outcomes. When nobody is at risk in the text and the speaker is describing someone else, nothing is detected and the action is monitor. When a person is at risk and a responsible adult in the text is already acting, an urgent action becomes flag_for_review with the severity and score brought down, and never lower. It never applies to a person speaking about their own risk, to a verdict supported by an acute marker, or when earlier turns are in play. Absent unless it applied.

protective_speech_capped_reasonstring

Human-readable explanation, naming which of the two outcomes applied. Present only when protective_speech_capped is true.

number_coincidence_clearedboolean

True when an alert rested on a number that is also known as a code, and a separate reading found the number in ordinary use (an address, a count, a price, a room) with nothing else harmful in the content. The verdict was cleared. Absent when this did not happen.

number_coincidence_cleared_reasonstring

Human-readable explanation. Present only when number_coincidence_cleared is true.

figurative_low_mood_cappedboolean

True when a review alert whose ONLY concern was low mood was lowered to monitor because a separate reading found the words were not meant literally: an exaggeration or an everyday expression about a task, a person or an event ("I can't take this homework any more"), a quotation, or something past and resolved. It applies only when the detector named low mood and nothing else, with no acute marker and no earlier turns in play; self-harm, a threat or any other category and it is never asked. The categories, severity and score are unchanged. It never clears to none. Absent unless it applied.

figurative_low_mood_capped_reasonstring

Human-readable explanation. Present only when figurative_low_mood_capped is true.

banter_cappedboolean

True when an alert that a rule for declared minors had raised was lowered ONE step because the text is banter between people who know each other: every strong word is an exaggeration about an everyday frustration (homework, a chore, a game), the other person answers in kind, nobody asks for it to stop or shows fear or hurt, nothing is kept from an adult beyond the trivial matter itself, and nobody says how they feel about themselves or their life. A raise to flag_for_review is lowered to monitor, with unsafe false. A raise to immediate_intervention is lowered to flag_for_review and no further, with everything else in the result unchanged, so a person still looks. It applies only when the analysis named no acute marker, its own action was lower than the one returned, and two separate readings agree; any other answer, or no answer, keeps the alert. banter_capped_reason says which step was taken. Absent when it did not apply.

banter_capped_reasonstring

Human-readable explanation. Present only when banter_capped is true.

no_child_shown_cappedboolean

True when the request declared no age group and a review alert was lowered to monitor because nothing in the message shows a child. It applies only when every category named is one that ordinary adult life is mistaken for (sexual content, risky behaviour, substance use), with no acute marker and no earlier turns in play, and a separate reading found the writer mentioning their own everyday life, with no sign of anyone under 18 and nobody hurt, pressured or in danger. A declared age group, a message about another person, or any other category, and it is never applied. The categories, severity and score are unchanged. It never clears to none. Absent unless it applied.

no_child_shown_capped_reasonstring

Human-readable explanation. Present only when no_child_shown_capped is true.

conversation_distress_surfacedboolean

True when earlier turns were submitted, the analysis found nothing in the latest message, and a separate reading of the whole conversation found that someone expressed distress about their own life in an earlier turn (surfaced as distress_context at monitor) or said something about ending their life or hurting themselves (surfaced as self_harm for review). Only ever surfaces; never lowers a result. Absent otherwise.

conversation_distress_surfaced_reasonstring

Human-readable explanation. Present only when conversation_distress_surfaced is true.

logistics_only_cappedboolean

True when the only concern named was that a location or an arrangement was given (an address, a place, a time), no acute marker was present, and a separate reading found practical detail for an ordinary purpose and nothing else in the message. A location is not unsafe on its own, so the verdict is cleared (unsafe false, recommended_action none). Absent when it did not apply.

logistics_only_capped_reasonstring

Human-readable explanation. Present only when logistics_only_capped is true.

external_idstring

Echo of the external_id you provided in the request

customer_idstring

Echo of the customer_id you provided in the request

metadataobject

Echo of the metadata you provided in the request

Changes

    • warning

      removed the optional property from the response with the status

    • warning

      added the new block enum value to the response property for the response status

    • warning

      added the new flag_for_review enum value to the response property for the response status

    • warning

      added the new immediate_intervention enum value to the response property for the response status

    • warning

      added the new monitor enum value to the response property for the response status

    • warning

      added the new none enum value to the response property for the response status

    • info

      added the new optional request property /

    • info

      added the new optional request property /

    • info

      added the new optional request property

    • info

      added the new optional request property

    • info

      added the new optional request property

    • info

      added the new optional request property

    • info

      added the new optional request property /

    • info

      added the new optional request property /

    • info

      added the new optional request property /

    • info

      added the new optional request property

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status