super::routes::local_inference

put/local-inference/models/{model_id}/settings

Request

  • The document declares no server URL.
  • Auth: none declared

Path parameters

model_idstring required

Request body

context_sizeinteger nullable
flash_attentionboolean nullable
frequency_penaltynumber float
max_output_tokensinteger nullable
n_batchinteger nullable
n_gpu_layersinteger nullable
n_threadsinteger nullable
native_tool_callingboolean
presence_penaltynumber float
repeat_last_ninteger
repeat_penaltynumber float
use_jinjaboolean
use_mlockboolean

Response

Settings updated

context_sizeinteger nullable
flash_attentionboolean nullable
frequency_penaltynumber float
max_output_tokensinteger nullable
n_batchinteger nullable
n_gpu_layersinteger nullable
n_threadsinteger nullable
native_tool_callingboolean
presence_penaltynumber float
repeat_last_ninteger
repeat_penaltynumber float
use_jinjaboolean
use_mlockboolean

Changes

No recorded changes to this operation across all 1 revision of this API.