gateway.openapi_Gateway

Create Deployment

post/v1/accounts/{account_id}/deployments

Path parameters

account_idstring required

The Account Id

Query parameters

disableAutoDeployboolean

By default, a deployment created with a currently undeployed base model will be deployed to this deployment. If true, this auto-deploy function is disabled.

disableSpeculativeDecodingboolean

By default, a deployment will use the speculative decoding settings from the base model. If true, this will disable speculative decoding.

deploymentIdstring

The ID of the deployment. If not specified, a random ID will be generated.

validateOnlyboolean

If true, this will not create the deployment, but will return the deployment that would be created.

skipShapeValidationboolean

By default, a deployment will ensure the deployment shape provided is validated. If true, we will not require the deployment shape to be validated.

skipImageTagValidationboolean

If true, skip the image tag policy validation that blocks certain image tags. This allows creating deployments with image tags that would otherwise be blocked.

Request body

namestring

displayNamestring

Human-readable display name of the deployment. e.g. "My Deployment" Must be fewer than 64 characters long.

descriptionstring

Description of the deployment.

createTimestring date-time

The creation time of the deployment.

expireTimestring date-time

Deprecated: This field is deprecated and no longer causes auto-deletion. The time at which this deployment will automatically be deleted.

purgeTimestring date-time

The time at which the resource will be hard deleted.

deleteTimestring date-time

The time at which the resource will be soft deleted.

CREATING: The deployment is still being created.
READY: The deployment is ready to be used.
DELETING: The deployment is being deleted.
FAILED: The deployment failed to be created. See the status field for additional details on why it failed.
UPDATING: There are in-progress updates happening with the deployment.
DELETED: The deployment is soft-deleted.

minReplicaCountinteger

The minimum number of replicas. If not specified, the default is 0.

maxReplicaCountinteger

The maximum number of replicas. If not specified, the default is max(min_replica_count, 1). May be set to 0 to downscale the deployment to 0.

maxWithRevocableReplicaCountinteger

max_with_revocable_replica_count is max replica count including revocable capacity. The max revocable capacity will be max_with_revocable_replica_count - max_replica_count.

desiredReplicaCountinteger

The desired number of replicas for this deployment. This represents the target replica count that the system is trying to achieve.

replicaCountinteger

baseModelstring required

acceleratorCountinteger

The number of accelerators used per replica. If not specified, the default is the estimated minimum required by the base model.

clusterstring

If set, this deployment is deployed to a cloud-premise cluster.

enableAddonsboolean

If true, PEFT addons are enabled for this deployment.

draftTokenCountinteger

The number of candidate tokens to generate per step for speculative decoding. Default is the base model's draft_token_count. Set CreateDeploymentRequest.disable_speculative_decoding to false to disable this behavior.

draftModelstring

The draft model name for speculative decoding. e.g. accounts/fireworks/models/my-draft-model If empty, speculative decoding using a draft model is disabled. Default is the base model's default_draft_model. Set CreateDeploymentRequest.disable_speculative_decoding to false to disable this behavior.

ngramSpeculationLengthinteger

The length of previous input sequence to be considered for N-gram speculation.

enableSessionAffinityboolean

Whether to apply sticky routing based on user field. Serverless will be set to true when creating deployment.

directRouteApiKeysstring[]

The set of API keys used to access the direct route deployment. If direct routing is not enabled, this field is unused.

numPeftDeviceCachedinteger

directRouteType'DIRECT_ROUTE_TYPE_UNSPECIFIED' | 'INTERNET' | 'GCP_PRIVATE_SERVICE_CONNECT' | 'AWS_PRIVATELINK'

directRouteHandlestring

The handle for calling a direct route. The meaning of the handle depends on the direct route type of the deployment: INTERNET -> The host name for accessing the deployment GCP_PRIVATE_SERVICE_CONNECT -> The service attachment name used to create the PSC endpoint. AWS_PRIVATELINK -> The service name used to create the VPC endpoint.

deploymentTemplatestring

The name of the deployment template to use for this deployment. Only available to enterprise accounts.

US_IOWA_1: GCP us-central1 (Iowa)
US_VIRGINIA_1: AWS us-east-1 (N. Virginia)
US_VIRGINIA_2: OCI us-ashburn-1 [HIDE_FROM_DOCS]
US_ILLINOIS_1: OCI us-chicago-1
AP_TOKYO_1: OCI ap-tokyo-1
EU_LONDON_1: OCI uk-london-1 [HIDE_FROM_DOCS]
US_ARIZONA_1: OCI us-phoenix-1
US_TEXAS_1: Lambda us-south-3 (C. Texas)
US_ILLINOIS_2: Lambda us-midwest-1 (Illinois)
EU_FRANKFURT_1: OCI eu-frankfurt-1
US_TEXAS_2: Lambda us-south-2 (N. Texas)
EU_PARIS_1: Nebius eu-west1 [HIDE_FROM_DOCS]
EU_HELSINKI_1: Nebius eu-north1 [HIDE_FROM_DOCS]
US_NEVADA_1: GCP us-west4 [HIDE_FROM_DOCS]
EU_ICELAND_1: Crusoe eu-iceland1
EU_ICELAND_2: Crusoe eu-iceland1 (network1)
US_WASHINGTON_1: Voltage Park us-pyl-1 (Detach audio cluster from control_plane)
US_WASHINGTON_2: Voltage Park us-seattle-2
EU_ICELAND_DEV_1: Crusoe eu-iceland1 (dev) [HIDE_FROM_DOCS]
US_WASHINGTON_3: Vultr Seattle 1
US_ARIZONA_2: Azure westus3 (Anysphere BYOC) [HIDE_FROM_DOCS]
AP_TOKYO_2: AWS ap-northeast-1
US_CALIFORNIA_1: AWS us-west-1 (N. California)
US_MISSOURI_1: Nebius us-central1 (Anysphere BYOC) [HIDE_FROM_DOCS]
US_UTAH_1: GCP us-west3 (Utah)
US_TEXAS_3: Crusoe us-southcentral1 [HIDE_FROM_DOCS]
US_ARIZONA_3: Coreweave us-west-04a-1 [HIDE_FROM_DOCS]
US_GEORGIA_1: DigitalOcean us-atl1
US_GEORGIA_2: Vultr Atlanta 1
US_WASHINGTON_4: Coreweave us-west-09b-1
US_GEORGIA_3: Alicloud us-southeast-1
NA_BRITISHCOLUMBIA_1: Fluidstack ca-west-1
US_GEORGIA_4: DigitalOcean us-atl1 MI350X
EU_ICELAND_3: Crusoe eu-iceland1 (Anysphere BYOC) [HIDE_FROM_DOCS]
US_OHIO_1: Lambda us-midwest-2 (Ohio)
US_NEWYORK_1: DigitalOcean nyc2

maxContextLengthinteger

The maximum context length supported by the model (context window). If set to 0 or not specified, the model's default maximum context length will be used.

updateTimestring date-time

The update time for the deployment.

disableDeploymentSizeValidationboolean

Whether the deployment size validation is disabled.

enableMtpboolean

If true, MTP is enabled for this deployment.

enableHotLoadboolean

Whether to use hot load for this deployment.

hotLoadBucketType'BUCKET_TYPE_UNSPECIFIED' | 'MINIO' | 'S3' | 'NEBIUS' | 'FW_HOSTED'

enableHotReloadLatestAddonboolean

Allows up to 1 addon at a time to be loaded, and will merge it into the base model.

deploymentShapestring

The name of the deployment shape that this deployment is using. On the server side, this will be replaced with the deployment shape version name.

activeModelVersionstring

The model version that is currently active and applied to running replicas of a deployment.

targetModelVersionstring

The target model version that is being rolled out to the deployment. In a ready steady state, the target model version is the same as the active model version.

hotLoadBucketUrlstring

pricingPlanIdstring

Optional pricing plan ID for custom billing configuration. If set, this deployment will use the pricing plan's billing rules instead of default billing behavior.

Response

A successful response.