Simulations

Validate performance accuracy against provider benchmarks

Compares the simulation's throughput and latency estimates against known real-world provider benchmarks. Returns a validation result indicating whether the simulated performance metrics are within the acceptable tolerance (±15%).

Skip semantics: Resources without a serviceFamily characteristic are skipped entirely. For resources that do have a serviceFamily, individual metrics (max_throughput, baseline_latency, max_connections) are skipped when no benchmark entry exists for that provider/metric combination. When all resource-metric pairs are skipped, validations is empty, summary.total is 0, and checked is false. In that state valid: true means "no failures among the zero pairs checked" — it does not mean the simulation's performance accuracy was verified. Always check checked: true before acting on valid.

Requires read scope and ownership.

get/simulations/{simulationId}/validate-performance-accuracy

Request

  • Base URL: https://www.cloudworldmodel.ai/api
  • URL: https://www.cloudworldmodel.ai/api/simulations/{simulationId}/validate-performance-accuracy
  • Auth: none required

Path parameters

simulationIdstring required

Simulation UUID

Response

Performance accuracy validation result

validboolean

Whether all checked resource-metric pairs are within the ±15% performance tolerance. Only meaningful when checked is true.

checkedboolean

true when at least one resource-metric pair had a matching performance benchmark and was evaluated. false means no comparable data was found — the simulation resources may lack serviceFamily, or no performance benchmark exists for the provider/metric combination. When checked is false, valid: true does not mean accuracy was verified.

Example response

{
  "valid": true,
  "checked": true,
  "validations": [
    {
      "resource": "web-server",
      "provider": "aws",
      "serviceFamily": "compute",
      "size": "m5.large",
      "metric": "max_throughput",
      "simulatedValue": 9800,
      "benchmarkValue": 10000,
      "deviation": 2,
      "withinThreshold": true,
      "unit": "RPS",
      "source": "AWS EC2 m5.large benchmark"
    }
  ],
  "summary": {
    "total": 4,
    "passed": 4,
    "avgDeviation": 1.5,
    "skippedCount": 2,
    "skippedReasons": [
      "\"cache-layer\": no serviceFamily set",
      "\"db-primary\" max_throughput: no benchmark for aws/database/db.r5.large"
    ]
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.