Submit a targeted node failure injection
Fails one resource in the simulation (marks it critical) and records an error event.
Quick inject-failure uses the resource's bounded failureParkSteps window (or criticalSteps when that field is omitted): the targeted compute/Kubernetes node is unavailable and receives no routed traffic during that window, then rejoins the routing pool automatically when organic cooldown can resume. A targeted database is also unavailable during its bounded park window. With no serving database and positive offered traffic, the next step reports 100% errors, zero throughput, and errorBreakdown.dbFailure: 100. If an AWS Aurora writer explicitly declares a healthy related replica (replicaOf the writer) via characteristics.auroraStandbyResourceId, the modeled first step remains interrupted; on the second step the standby takes over and metrics.databases[].auroraFailover reports the failed and standby resource IDs, phase, and success. Promotion removes the writer-outage penalty while the writer remains parked. multiAz or a second, unrelated database alone does not prove a write-capable standby. Each quick step is one modeled simulation second: the second-step promotion is one second after injection, not 30 seconds. Chaos database_crash instead samples at 10-second intervals with a default 30-second promotion and a 1800-second sole-writer restart after the injection duration. Compare equivalent outage/serving phases, not equal step indices or wall-clock times. These are deterministic assumptions, not observed AWS failover timing. Its original connection configuration is restored after the window, followed by organic cooldown, or earlier via recover-resource. Random selection remains compute/Kubernetes-only. Network and other non-node resources cannot be quick-failed. Authenticated callers can instead use failure.create with database_overload for a typed database failure; this typed tool is not available anonymously. This is different from the persistent reversible instance_down failure created through /failures, which remains unavailable until deactivated or recovered with recover-resource; instance_kill permanently removes the resource. Poll the step response or GET /simulations/{simulationId} for availabilityState, isRoutable, routedRps, and recoveryProgress. failureParkSteps must be a positive integer; an explicit zero is rejected rather than treated as an ambiguous no-park mode.
Targeting (recommended) — pass resourceId or resourceName (human-readable name; exact match preferred, an unambiguous prefix is accepted) in the request body to fail a specific, named resource (e.g. { "resourceName": "web-server-01" }). This path is deterministic and reproducible — use it for controlled scenarios, CI fault-tree runs, and any case where a predictable outcome matters. If resourceName matches more than one resource, a 400 is returned with a candidates list naming every match so the caller can retry unambiguously. If the resolved resource is not in a faileable state (already critical/warning), a 400 describing its current status is returned.
Random mode (opt-in) — pass {"random": true} to explicitly select a random healthy node via the simulation's RNG. This path is non-deterministic and not suitable for controlled scenarios or CI replay. Use it only when intentional random selection is explicitly desired. An empty body {} returns 400 with a hint naming all three valid patterns.
The response always echoes the applied outcome via resolvedResourceId, resolvedResourceName, and previousHealth — on the random path these are populated from whichever resource was selected.
Returns 400 if no healthy nodes are available to fail. For fine-grained control over failure type and duration, use POST /simulations/{simulationId}/failures instead.
Resource name discovery (agents: do this first) — before calling this endpoint, call GET /simulations/{simulationId} and read the top-level resources array. Each element has an id and a name field. Use one of those values as resourceId or resourceName in your request body. The response also returns a healthyNodes array [{id, name}, ...] listing every currently healthy resource, so you can update your inventory on each call without a separate lookup.
Unknown fields are rejected before payment settlement — if the request body contains any field other than resourceId, resourceName, or random (for example the common agent mistake targetResourceId, target, or resource), the endpoint returns a structured 400 with a "Did you mean" hint before any USDC charge is settled. An empty body {} also returns 400 with a hint naming all three valid patterns.
Requires write scope and ownership.
Request
- Base URL: https://www.cloudworldmodel.ai/api
- URL: https://www.cloudworldmodel.ai/api/simulations/{simulationId}/inject-failure
- Auth: one of:
- HTTP bearer
- API key in header X-PAYMENT
- API key in header PAYMENT-SIGNATURE
Path parameters
Simulation UUID
Request body
Response
Node failure injected
Example response
{
"simulation": {
"resources": [
{
"id": "ec2-1",
"type": "compute",
"name": "Web Server 1",
"provider": "aws",
"location": {
"regionKey": "us-east-2"
},
"characteristics": {
"workload": "crud",
"appRuntime": "node",
"appWorkerCount": 2,
"appDbPoolSize": 250,
"workloadDatabaseVersion": "8.0",
"workloadDatabaseEngine": "mysql",
"rdsSqlServerBilling": {
"instanceClass": "db.t3.micro",
"storageType": "gp2",
"allocatedStorageGiB": 20,
"durationHours": 480,
"sustainedCpuPercent": 43
},
"serviceFamily": "ec2",
"containerCpu": 0.5,
"containerMemoryGiB": 1,
"runtimeMemoryProfile": {
"workingSetGiB": 0.78125
},
"concurrency": 80,
"requestDurationSeconds": 0.25,
"requestServingTargetUtilizationPct": 70,
"billingState": "stopped",
"size": "m5.large",
"instanceType": "t3.medium",
"ocpus": 4,
"maxThroughput": 2000,
"capacityRps": 100,
"baseLatency": 3,
"maxConnections": 800,
"capacityGB": 500,
"maxIops": 3000,
"inferenceMode": true,
"accelerator": "a10g",
"managedComputePlatform": "nvidia-dgx-cloud",
"openshiftOffering": "rosa-hcp",
"topology": {
"intraNode": "nvlink-nvswitch",
"interNode": "infiniband"
},
"intraNode": "nvlink-nvswitch",
"interNode": "infiniband",
"includesControlPlane": true,
"bundledNetworking": true,
"trainingMode": true,
"tokensPerRequest": 500,
"inputTokensPerRequest": 1000,
"cacheHitRate": 0.85,
"cacheableRequestShare": 0.75,
"asynchronousRequestShare": 0.15,
"queueConsumerCount": 4,
"queueDrainRatePerConsumerRps": 125,
"queueCapacityMessages": 20000,
"queueSimulationStepSeconds": 1,
"autoscaling": true,
"maxInstances": 5,
"minInstances": 2,
"scaleOutCpuThreshold": 60,
"scaleInCpuThreshold": 25,
"nodeCount": 2,
"workerVcpu": 4,
"minNodes": 2,
"maxNodes": 10,
"perNodeRate": 3.2,
"pricingModel": "on-demand",
"nodePools": [
{
"name": "general",
"nodeCount": 2,
"workerVcpu": 8,
"minNodes": 2,
"maxNodes": 10,
"perNodeRate": 0.096,
"maxThroughput": 10000
}
],
"streamShardsCount": 4,
"streamWriteRps": 3500,
"eventNotificationsEnabled": true,
"partitionCount": 8,
"streamPartitions": 4,
"partitionWriteRps": 5000,
"warmupSteps": 5,
"warmupSeverity": 0.8
},
"routedRps": 15,
"availabilityState": "available",
"isRoutable": true,
"failureParkStepsRemaining": 3,
"recoveryPolicy": {
"criticalCpuThreshold": 75,
"criticalSteps": 2,
"warningCpuThreshold": 60,
"warningSteps": 1,
"failureParkSteps": 2
}
}
],
"normalizedConfig": {
"resources": [
{
"rdsSqlServerBilling": {
"input": {
"instanceClass": "db.t3.micro",
"storageType": "gp2",
"allocatedStorageGiB": 20,
"durationHours": 480,
"sustainedCpuPercent": 43
}
},
"rateProvenance": {
"sku": "m7g.large",
"source": "AWS EC2 Price List API (Linux, shared tenancy, on-demand)"
},
"dynamoDbBilling": {
"referenceRequestRateRps": 50000
},
"requestServing": {
"serviceFamily": "lambda"
},
"behaviorModel": {
"latencyPath": "gpu-inference/ttft-decode",
"description": "Token-economics model: TTFT + decode, weighted by throughput and batch pressure.",
"modelingNote": "TTFT and decode-throughput figures are CWM simulation-model estimates, not externally measured hardware benchmarks.",
"topologyThroughputFactor": 0.92,
"topologyThroughputCalibrationStatus": "estimated",
"ibNetworkingCostPerNodePerHour": 2
},
"inferenceWarnings": [
"Resource has inference/GPU characteristics (accelerator: h100, tokensPerRequest: 512) but inferenceMode is not enabled. The generic-kubernetes/load-saturation latency model will be used. Set characteristics.inferenceMode: true to activate gpu-inference/ttft-decode."
]
}
]
},
"resilienceConfig": {
"dependencies": [
{
"id": "web-to-db",
"sourceId": "web-1",
"targetId": "db-1",
"requestRatio": 1,
"authDependencyId": "gateway-to-token-service",
"retryPolicy": {
"maxRetries": 3,
"backoffMs": 100,
"backoffMultiplier": 2,
"jitterRatio": 0.1,
"timeoutMs": 2000,
"retryBudgetRatio": 2,
"retryBudgetRps": 1000,
"retryActorId": "gateway_optimistic_retries"
},
"capacity": {
"maxRps": 10000,
"maxConcurrent": 200,
"meanServiceTimeMs": 100
},
"protection": {
"circuitBreaker": {
"failureRateThreshold": 0.5,
"minimumRequests": 20,
"openSteps": 3,
"halfOpenMaxRequests": 10
},
"rateLimitRps": 5000
}
}
],
"scheduledFaults": [
{
"id": "db-capacity-fault",
"targetResourceId": "db-1",
"dependencyId": "web-to-db",
"startStep": 5,
"endStep": 15,
"capacityPercent": 30,
"maxConcurrent": 5,
"addedLatencyMs": 500,
"errorRate": 0.3,
"trafficMultiplier": 2.25
}
],
"maxCascadeDepth": 4,
"maxGeneratedRps": 100000,
"maxStepWork": 512
}
}
}Changes
No recorded changes to this operation across all 1 revision of this API.