Sign in
ConfigGuesser

Analyze Model

Attempt to guess required GPU count and VRAM for a model on huggingface, assuming safetensors format.

get/guess/vllm_config

Query parameters

modelstring required

Response

Successful Response

total_model_sizeinteger required
required_gpusinteger required
min_vram_per_gpuinteger required
model_typestring required
quantizationstring nullable required
num_attention_headsinteger required
num_key_value_headsinteger nullable required
hidden_sizeinteger required
num_layersinteger required

Changes