Studio Chat Complete

An enumeration.

How many chat responses to generate. Range: 1 – 16 Notes:

• If n > 1, setting temperature=0 will fail because all answers are guaranteed to be duplicates.
• n must be 1 when stream = True

The maximum number of tokens to allow for each generated response message. Typically the best way to limit output length is by providing a length limit in the system prompt (for example, "limit your answers to three sentences"). Range: 0 – 4096

How much variation to provide in each answer. Setting this value to 0 guarantees the same response to the same question every time. Setting a higher value encourages more variation. Modifies the distribution from which tokens are sampled. More information Range: 0.0 – 2.0

Limit the pool of next tokens in each step to the top N percentile of possible tokens, where 1.0 means the pool of all possible tokens, and 0.01 means the pool of only the most likely next tokens. [More information] (https://docs.ai21.com/docs/sampling-from-language-models#topp) Range: 0 <= value <=1.0

Whether or not to stream the result one token at a time using server-sent events. This can be useful when waiting for long results where a long wait time for an answer can be problematic, such as a chatbot. If set to True, then n must be 1. A streaming response is different than the non-streaming response.