SCscaleapi

Get Model Endpoint

Changed on

Describe the LLM Model endpoint with given name.

get/v1/llm/model-endpoints/{model_endpoint_name}

Request

  • The document declares no server URL.
  • Auth
    • HTTP basic
    • or OAuth 2
      Password
      Token URL: token

Path parameters

model_endpoint_namestring required

Response

Successful Response

idstring required
namestring required
model_namestring required
source'hugging_face' required
status'READY' | 'UPDATE_PENDING' | 'UPDATE_IN_PROGRESS' | 'UPDATE_FAILED' | 'DELETE_IN_PROGRESS' required
inference_framework'deepspeed' | 'text_generation_inference' | 'vllm' | 'lightllm' | 'tensorrt_llm' | 'sglang' required
inference_framework_image_tagstring nullable
num_shardsinteger nullable
quantize'bitsandbytes' | 'awq'
checkpoint_pathstring nullable
chat_template_overridestring nullable

A Jinja template to use for this endpoint. If not provided, will use the chat template from the checkpoint

task_expires_secondsinteger nullable

For async endpoints, how long a task can wait in queue before expiring (in seconds).

queue_message_timeout_secondsinteger nullable

For async endpoints, the queue message visibility/lock timeout in seconds (SQS VisibilityTimeout / ASB lock_duration). Note: Azure Service Bus has a maximum of 300 seconds; values above this will be clamped.

Changes

  • ea1812999de9Revision changes
    • warning

      removed the optional property // from the response with the status

    • warning

      removed the optional property // from the response with the status

    • warning

      removed the optional property from the response with the status

    • warning

      removed the optional property //// from the response with the status

    • warning

      removed the optional property // from the response with the status

    • warning

      removed the optional property // from the response with the status

    • warning

      removed the optional property from the response with the status