Test Model Connection

Test a direct connection to a specific model.

This endpoint allows you to verify if your proxy can successfully connect to a specific model. It's useful for troubleshooting model connectivity issues without going through the full proxy routing.

Example:

# If model is configured in proxy_config.yaml, you only need to specify the model name:
curl -X POST 'http://localhost:4000/health/test_connection' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "litellm_params": {
        "model": "gpt-4o"
    },
    "mode": "chat"
  }'

# The endpoint will automatically use api_key, api_base, etc. from proxy_config.yaml

# You can also override specific params or test with custom credentials:
curl -X POST 'http://localhost:4000/health/test_connection' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "litellm_params": {
        "model": "azure/gpt-4o",
        "api_key": "os.environ/AZURE_OPENAI_API_KEY",
        "api_base": "os.environ/AZURE_OPENAI_ENDPOINT",
        "api_version": "2024-10-21"
    },
    "mode": "chat"
  }'

Note:

• If the model is configured in proxy_config.yaml, credentials (api_key, api_base, etc.) will be automatically loaded from the config (with resolved environment variables).
• A request naming a stored credential (litellm_credential_name) that the configuration does not name is probed with that credential instead, and inherits no credentials from the configuration its model string happened to match.
• You can override specific params by including them in the request.
• You can use os.environ/VARIABLE_NAME syntax to reference environment variables, which will be resolved automatically (same as in proxy_config.yaml).

Returns: dict: A dictionary containing the health check result with either success information or error details.