--- title: "Modify an existing cluster. Updates a cluster with the given data. Note that most of the data in the cluster is read-only and automatically populated by the local cluster agents." method: PUT path: "/v2/organizations/{organization}/clusters/{cluster}" tags: ["Clusters"] --- # Modify an existing cluster. Updates a cluster with the given data. Note that most of the data in the cluster is read-only and automatically populated by the local cluster agents. `PUT /v2/organizations/{organization}/clusters/{cluster}` ## Request - Base URL: `https://docs.tetrate.io` (the document declares no server; this is the origin it was published from) - URL: `https://docs.tetrate.io/v2/organizations/{organization}/clusters/{cluster}` - Auth: none declared - TypeScript: (a function with typed inputs; its response is checked against the schema) ## Path parameters - `organization` string, required - `cluster` string, required ## Request body - object: A Kubernetes cluster managing both pods and VMs. - `description` string - `displayName` string - `etag` string - `installTemplate` ClusterInstallTemplate: InstallTemplate provides templates ready to be used in the ControlPlane (cluster onboard) installation. - `message` string - `helm` V1alpha1Values: Values available for the TSB Control Plane chart. This is an alpha API, so future versions could include breaking changes. - `image` Commonv1alpha1Image: Values for the TSB operator image. - `registry` string: Registry used to download the operator image. - `tag` string: The tag of the operator image. - `spec` V1alpha1ControlPlaneSpec: ControlPlaneSpec defines the desired installed state of control plane components. Specifying a minimal ControlPlaneSpec with hub, clusterName, and managementPlane set will create an installation with sensible defaults. - `hub` string, required: TSB container hub path e.g. docker.io/tetrate. - `imagePullSecrets` InstallkubernetesLocalObjectReference[]: Pull secrets can be specified globally for all components, or defined into the `kubeSpec.serviceAccount` of every component if needed. In case both are defined, the most specific one (the one defined at the component) level is used. List of references to secrets in the same namespace to use for pulling any images in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet. More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#service_account-v1-core - `name` string: Name of the referent. - `components` V1alpha1ControlPlaneComponentSet: The set of components that make up the control plane. Use this to override application settings or Kubernetes settings for each individual component. - `collector` V1alpha1OpenTelemetryCollector: Kubernetes settings for the OpenTelemetryCollector component. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevel` string: Specifies the log level for OTEL collector component. Supported log level: "debug", "info", "warn", "error", "dpanic", "panic", and "fatal". - `oap` V1alpha1Oap: Kubernetes settings for the OAP (SkyWalking) component. - `streamingLogEnabled` boolean: Feature flag to determine whether on-demand streaming logs should be enabled. - `onDemandEnvoyMetricsEnabled` boolean: Feature flag to determine whether on-demand envoy metrics should be enabled. If enabled, the envoy proxy will provide a set of metrics that can be queried using the metrics service. OAP will provide a query API that can be used to collect envoy proxy metrics for specific pods. This is only for temporary and real-time queries that can be used, for example, for application troubleshooting use cases. These metrics are not persisted. - `storageIndexMergingEnabled` boolean: Feature flag to determine whether metrics/meter and records should be shard into multi-physical indices, or instead if they should be merged into a single physical index. By default "false", metric/meter and records are sharded into multi-physical indices. Instead of sharding, if enabled by setting it to "true", metrics/meter and records will be merged into one physical index template `metrics-all` and `records-all`. This feature flag must be set on all clusters and have the same value as the management plane's one, otherwise control plane observability data could be written to the wrong or not existing index. In this storage mode, user can adjust each concrete index should have to scale out by setting `storageSpecificIndexSettings` field in the management plane install manifest. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevel` string: Specifies the log level for OAP component. Supported log level: "all", "debug", "info", "warn", "error", "fatal", "off" and "trace". - `xcp` Controlplanev1alpha1XCP: Kubernetes settings for the XCP component. - `centralAuthMode` 'UNKNOWN' | 'MUTUAL_TLS' | 'JWT': - UNKNOWN: Default when unset, do not use - MUTUAL_TLS: GRPC stream is encrypted with mutual TLS - JWT: XCP Edges present a JWT bearer token in the GRPC headers - `configProtection` CommonConfigProtection - … - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `revision` string: $hide_from_docs Configures the istio revision tag. If configured, istio upgrade will not be in-place upgrade. A new istio control plane with the configured revision will be deployed. Selectively sidecars and gateways could be moved to newer control plane. Note that it is not the istio version. Istio version is fixed for a particular tsb version and that is not a configurable setting. Revision should be configured to human readable value for example tsb-1-5. For further reference, https://istio.io/latest/blog/2020/multiple-control-planes/#configuring Deprecated: replaced by isolation_boundaries, where each isolation boundary can have mulitple revisions. - `isolationBoundaries` V1alpha1IsolationBoundary[]: Configures Isolated Istio environments along with Istio revisions for each environment. IsolationBoundaries can be empty when the feature flag IstioIsolationBoundaries is disabled. Once enabled, isolation boundaries can be configured. - … - `enableHttpMeshInternalIdentityPropagation` boolean: Enables HTTP mesh internal service identity propagation across gateway hops, utilizing the propagated identity for evaluating TSB RBAC rules. Users should enable this feature when they want to create RBAC rules around request's origin client identity for east west traffic. The most common case for this would be when using authorization features such as ALLOW/DENY rules mode and ServiceSecuritySettings in cross-cluster environment. This feature is disabled by default. - `centralProvidedCaCert` boolean: If true, obtain the CA cert for Istio from XCP central. To enable it, the XCP Central needs to be configured with `certIssuer.clusterIntermediateCASettings: {}`. - `logLevels` object: Loglevel for XCP. Supported log level: "none", "fatal", "error", "warn", "info", "debug". - `istio` V1alpha1Istio: Mesh and Kubernetes settings for Istio. - `tsbVersion` string: Specifies the tsb release version. This is used by the tsb control plane operator in determining the xcp version, which would eventually decide Istio version. If not provided explicitly, this defaults to the current tsb version. - `mountInternalWasmExtensions` boolean: When this flag is set, the TSB internal WASM extensions will be mounted into the Sidecar, Ingress and Egress gateway pods automatically. These extensions will be loaded as local files instead of being downloaded from a remote OCI registry or HTTP endpoint. This is currently disabled by default, but in future releases this flag will be enabled by default. - `kubeSpec` KubernetesKubernetesIstioComponentSpec: KubernetesIstioComponentSpec is the common set of Kubernetes resource configuration for Istio. It differs from the standard component specs in that it supports CNI configuration. - … - `traceSamplingRate` number, double: The percentage of traces Envoy will sample. - `defaultWorkloadCertTTL` string: The default TTL of issued workload certificates. This sets both the default client-side CSR TTL and the default server-side issued certificate TTL. - `maxWorkloadCertTTL` string: The maximum TTL that can be set in issued workload certificates. - `trustDomain` string: The trust domain corresponds to the trust root of a system. Refer to [SPIFFE-ID](https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE-ID.md#21-trust-domain). If omitted, TSB will configure the trust domain as `CLUSTER_NAME.tsb.local`, where `CLUSTER_NAME` is the name of the cluster object in TSB for this control plane. - `baseOverlays` V1alpha1K8sObjectOverlay[]: The overlays applied to the Istio base component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in `kubeSpec.overlays` are ignored. - … - `pilotOverlays` V1alpha1K8sObjectOverlay[]: The overlays applied to the Istio pilot component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in `kubeSpec.overlays` are ignored. - … - `cniOverlays` V1alpha1K8sObjectOverlay[]: The overlays applied to the Istio CNI component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in `kubeSpec.overlays` are ignored. - … - `logLevels` object: Specifies the global logging level settings for the Istio control plane components. - `rateLimitServer` V1alpha1RateLimitServer - `backend` RateLimitServerBackend, required: External Backend Database types. This points to the backend used by the ratelimit server as a key/value store. - … - `domain` string: The domain field allows ratelimits to be namespaced to a certain domain. To support common ratelimits across multiple clusters set this string to a common value, across them. This assumes that the same backend (uri) is being used. By default the domain is set to the name of the control plane cluster. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `hpaAdapter` V1alpha1HpaAdapter: Kubernetes settings for the OAP (SkyWalking) HPA adapter component. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `onboarding` V1alpha1Onboarding: Settings for the `Workload Onboarding` component. - `operator` V1alpha1OnboardingOperator: Kubernetes settings for the `Workload Onboarding Operator` component. - … - `repository` V1alpha1OnboardingRepository: Kubernetes settings for the `Workload Onboarding Repository` component. - … - `plane` Controlplanev1alpha1OnboardingPlane: Configure `Workload Onboarding Plane` component. - … - `satellite` V1alpha1Satellite: Kubernetes settings for the Satellite (SkyWalking-Satellite) component. - `enabled` boolean: Satellite is an optional component. If enabled is false, this component will not be installed. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevel` string: Specifies the log level for the component. Supported log level: "panic", "fatal", "info", "warn", "error", "debug" and "trace". - `ngac` V1alpha1NGAC: Kubernetes settings for the NGAC component. - `enabled` boolean: NGAC is an experimental component. If enabled is false, this component will not be installed. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevels` object: The log level configuration by scopes. Supported log level: "none", "error", "info", "debug". - `gitops` CommonGitOps: The GitOps component configures the features that allow integrating the Management Plane and/or the Control Plane cluster with Continuous Deployment pipelines. - `enabled` boolean: The GitOps component is in beta and disabled by default. If Management and Control Planes are installed in the same cluster, Continuous Deployment Integration should only be enabled in one of both planes. However, if the GitOps component is enabled in both planes, only the Control Plane GitOps component will remain enabled. The Management Plane GitOps component will not be enabled, even though it is explicitly enabled. - `reconcileInterval` string: Interval at which the reconcile process will run. The reconcile process will read all TSB CRs that exist in the cluster and reapply them to the management plane, to make sure the cluster CRs remain as the source of truth. Format: 1h/1m/1s/1ms. A value of 0 disables the reconcile loop. Default: 10m. - `batchWindow` string: When configured, all admission requests will be paused for the configured duration. Once the window interval is closed, all paused admission requests will be sent together to the Management Plane as a single request. Batching of requests is disabled by default and should be enabled only if there is high concurrency and ordering of resources could be an issue. By configuring a batch window the concurrency and ordering issues may be mitigated, although it will introduce a constant latency to all requests of the configured time window. When enabled, it is recommended to use a small value, for example 1 second. - `managementplaneRequestTimeout` string: The GitOps component performs operations against the management plane through the k8s webhook. This allows configuring the duration of each operation in order to fail early if it takes too much. This value cannot be lower than `webhook_timeout` due to the request being tied to the ones received by the k8s webhook. Format: 1h/1m/1s/1m. Any value <= 0 will be reset to the default value. Default: 25s. - `reconcileRequestTimeout` string: The GitOps component performs operations against the management plane internal reconcile loop. This allows configuring the duration of each operation to fail early if it takes too long. Format: 1h/1m/1s/1m. Any value <= 0 will be reset to the default value. Default: 2m. - `webhookTimeout` string: Timeout that will be set in the k8s gitops webhook resource. Format: 1h/1m/1s/1m. Default: 30s. Allowed values must be between 0s and 30s. - `internalCertProvider` CommonInternalCertProvider: InternalCertProvider describes the certificate provider configuration for TSB internal purposes like kubernetes webhook certificate. TSB supports cert-manager out of the box. - `certManager` CommonCertManagerSettings: CertManagerSettings represents the settings used for the cert-manager installation. TSB supports installing and managing the lifecycle of the cert-manager installation. - … - `custom` CommonCustomCertProviderSettings: CustomCertProviderSettings represents the settings used for the custom certificate provider. Users can configure the CSR signer required for certificate signing and point to the CA bundle to be used to validate the certificates. - … - `defaultKubeSpec` KubernetesKubernetesSpec: KubernetesSpec is a common set of Kubernetes resource configuration for the install CRs, that will be common to all of its components. - `deployment` KubernetesGlobalDeployment - … - `service` KubernetesGlobalService - … - `account` InstallkubernetesServiceAccount - … - `job` KubernetesGlobalJob - … - `wasmfetcher` V1alpha1WASMFetcher: Settings for the WASM Fetcher component. - `cacheDisableInsecureRegistries` boolean: Denies insecure registries to be used for fetching WASM modules. Defaults to `false`. - `cacheExpiration` string: WASM Module cache expiration time. Defaults to `24h`. - `cacheMaxRetries` integer: Maximum number of retries when fetching WASM modules from the OCI registry. Defaults to `5`. - `cachePurgeInterval` string: WASM cache purge interval to periodically clean up the stale WASM modules. Defaults to `1h`. - `cacheRequestTimeout` string: Specifies the timeout used when retrieving the WASM plugin from the OCI registry. Defaults to `15s`. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevels` object: The log level configuration by scopes. Supported log levels: "none", "error", "info", "debug". - `defaultLogLevel` string: The default log level for all components if the per component log level config is not specified. Note that the supported log level for different components can be different. - `route53Controller` V1alpha1Route53Controller: Kubernetes settings for the Route53 Integration Controller component. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `providerSettings` V1alpha1ProviderSettings: Configure Kubernetes provider specific settings. For example to configure EKS to use network load balancers (NLB) by default: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: providerSettings: eks: useNlbByDefault: true ``` To configure Route53 the only option that you must specify is the Service Account name to use for IAM role. You should create the Service Account before enabling the Route53 integration controller. You can do that using `eksctl`. Example: ```bash SA_NAME=route53-controller CP_NAMESPACE=istio-system eksctl create iamserviceaccount \ --cluster $EKS_CLUSTER_NAME \ --name $SA_NAME \ --namespace $CP_NAMESPACE \ --attach-policy-arn $POLICY_ARN \ --approve ``` where: * $EKS_CLUSTER_NAME is the name of the EKS cluster. * $SA_NAME is the name of the Service Account to create. * $CP_NAMESPACE is the namespace where the Control Plane is installed. Usually istio-system. * $POLICY_ARN is the ARN of the policy to attach to the Service Account - the policy should allow the Service Account to manage Route53 resources. More details can be found in the [Publishing a Service docs](https://docs-preview.tetrate.io/service-express/Tech-Preview/getting-started/publish-service) After creating the Service Account you can enable the Route53 integration controller using the following configuration: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: providerSettings: route53: serviceAccountName: $SA_NAME ``` - `eks` V1alpha1EKSSettings: Settings specific to Elastic Kubernetes Service (EKS). - `useNlbByDefault` boolean: When true, gateways will be configured to use NLBs with cross zone load balancing enabled when the load balancer type is not configured. When false, no additional annotations will be added. - `route53` V1alpha1Route53Settings: Settings for integration with Route53 service. - `serviceAccountName` string, required: Service account name to use for IAM role. Required. - `namespaceSelector` V1alpha1Route53SettingsNamespaceSelector: NamespaceSelector specifies which namespaces controller will watch. - … - `policy` 'SYNC' | 'UPSERT_ONLY' | 'CREATE_ONLY': Policy that defines how DNS records are managed. - SYNC: Allow full synchronization. - UPSERT_ONLY: Don't allow delete DNS records. - CREATE_ONLY: Allow only creating DNS records. - `domainFilter` string[]: Limit possible target zones by a domain suffix; specify multiple times for multiple domains. Optional. - `interval` string: Duration of interval between individual synchronizations. Default: 60s. - `ttl` string, int64: Default TTL (in seconds) value for DNS records. Default: 300. - `evaluateTargetHealth` boolean: Control whether to evaluate the health of a DNS target. Default: true. - `filterSettings` Route53SettingsFilterSettings: Filter settings for route53 controller. - … - `managementPlane` V1alpha1ManagementPlaneSettings, required: Configure the management plane connection. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: managementPlane: host: tsb.tetrate.io port: 8443 selfSigned: true clusterName: control-plane-cluster ``` - `host` string, required: Management plane host address (can be hostname or IPv4/IPv6 address). - `port` integer, required: Port management plane is listening on. - `selfSigned` boolean: Management plane uses a self signed or private TLS certificate. If true, the CA bundle used to verify the MP's TLS certificate must be in a secret `mp-certs` under the key `ca.crt`. - `clusterName` string, required: The name of the Cluster object that was created in the Management Plane representing this Control Plane cluster. - `meshExpansion` V1alpha1MeshExpansionSettings: Configure mesh expansion to connect workloads external to Kubernetes to the mesh. To enable mesh expansion set it to an empty object: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: {} ``` If external workloads are unable to communicate with the default mesh expansion gateway via external IPs or hostnames, then you must specify the gateway that enables them to do so. This custom gateway must be configured to forward this communication to the VM gateway service: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: customGateway: host: customgateway.tetrate.io port: 15443 ``` To automate onboarding of workloads from auto-scaling groups of VMs, you need to enable the `Workload Onboarding Plane`. `Workload Onboarding Agent`, a component that you install next to the workload, will connect to the `Workload Onboarding Plane` to authenticate itself, ask permission to join the mesh, register the workload into the mesh and retrieve boot configuration required to start `Istio Sidecar`. All communication between the `Workload Onboarding Agent` and the `Workload Onboarding Plane` must occur over TLS. Therefore, to enable `Workload Onboarding Plane` you must provide a TLS certificate for the endpoint that exposes `Workload Onboarding API` to `Workload Onboarding Agents`. Make sure that TLS certificate is signed by the certificate authority known to `Workload Onboarding Agents`. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: onboarding: endpoint: hosts: - onboarding.example.org secretName: onboarding-tls-cert tokenIssuer: jwt: expiration: 1h localRepository: {} ``` To onboard workloads from custom on-premise environments, you can leverage support for [OIDC ID Tokens](https://openid.net/specs/openid-connect-core-1_0.html#IDToken). If workloads in your custom environment can authenticate themselves by means of an [OIDC ID Token](https://openid.net/specs/openid-connect-core-1_0.html#IDToken), you can define a list of JWT issuers permitted by the `Workload Onboarding Plane`. For example, ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: onboarding: endpoint: hosts: - onboarding.example.org secretName: onboarding-tls-cert localRepository: {} workloads: authentication: jwt: issuers: - issuer: "https://mycompany.corp" jwksUri: "https://mycompany.corp/jwks.json" shortName: "mycorp" tokenFields: attributes: jsonPath: .custom_attributes ``` To ensure there will be no traffic loss when an onboarded workload gets shutdown, you can configure the time period to delay the shutdown for after deregistering the workload from the mesh, which will give enough time to reconfigure all affected mesh nodes to not load balance requests to the deregistered workload before it becomes unavailable. For example, ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: onboarding: endpoint: hosts: - onboarding.example.org secretName: onboarding-tls-cert localRepository: {} workloads: deregistration: propagationDelay: 15s ``` - `customGateway` MeshExpansionSettingsGateway: A custom mesh expansion gateway. This is required when the workload can't access the default gateway directly via the external IP or hostname. - `host` string, required: Mesh expansion gateway host address (can be hostname or IP address). - `port` integer, required: Port mesh expansion gateway is listening on. - `onboarding` V1alpha1MeshExpansionSettingsOnboardingPlane: Configuration of the `Workload Onboarding Plane`. - `uid` string: Unique identifier of this particular installation of the `Workload Onboarding Plane`. Is used in the workload authentication flow to prevent replay attacks that abuse compromised workload credentials intended for a different installation of the `Workload Onboarding Plane`. Defaults to an auto-generated UUID. - `endpoint` OnboardingPlaneEndpoint, required: Configuration of the endpoint exposing `Workload Onboarding API` to `Workload Onboarding Agents`. - … - `tokenIssuer` OnboardingPlaneTokenIssuer: Configuration of the built-in `Workload Onboarding Token Issuer`. - … - `localRepository` OnboardingPlaneLocalRepository: Configuration of the local repository with `DEB` and `RPM` packages of the `Workload Onboarding Agent` and `Istio Sidecar`. - `workloads` V1alpha1WorkloadConfiguration: WorkloadConfiguration specifies configuration of the workload handling. - … - `telemetryStore` ControlPlaneSpecTelemetryStore, required: Configure the store that TSB will use to persist application telemetry data. Select one of the `TelemetryStore` settings to see complete examples. - `elastic` V1alpha1ElasticSearchSettings: Configure an Elasticsearch connection. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: telemetryStore: elastic: host: elastic port: 5678 protocol: https selfSigned: true version: 7 ``` - `host` string, required: Elasticsearch host address (can be hostname or IP address). - `port` integer, required: Port Elasticsearch is listening on. - `protocol` 'https' | 'http': The list of supported protocols to communicate with Elasticsearch. - `selfSigned` boolean: Use Self-Signed certificates. The Self-signed CA bundle and key must be in a secret called es-certs. - `version` integer: DEPRECATED: Major version of the Elasticsearch cluster. Currently supported Elasticsearch major versions are `6`, `7`, and `8`. - `meshObservability` ControlPlaneSpecMeshObservability: Configure how the mesh should be observed, which observability functionalities should be enabled to observe your registered services in the mesh, and the store properties that TSB will use to persist application observability data like metrics, traces, logs. If omitted, the operator will assume a demo installation and for your convenience install a demo grade mesh observability setting. Select one of the `MeshObservability` settings to see complete examples. - `demoSettings` CommonMeshObservabilitySettings: Configure mesh observability. The following examples enable the analysis and generation of RED metrics for each endpoint of your registered services. Notice that both, ManagementPlane and ControlPlane, need to be aligned with this configuration. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ManagementPlane metadata: name: managementplane spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` - `apiEndpointMetricsEnabled` boolean: Toggle to process, analyze, and generate api endpoints RED metrics. By default `false` which means disabled. If you want to analyze all your request and generate RED metrics for each endpoint of your registered services in the mesh, set it to `true`. - `settings` CommonMeshObservabilitySettings: Configure mesh observability. The following examples enable the analysis and generation of RED metrics for each endpoint of your registered services. Notice that both, ManagementPlane and ControlPlane, need to be aligned with this configuration. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ManagementPlane metadata: name: managementplane spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` - `apiEndpointMetricsEnabled` boolean: Toggle to process, analyze, and generate api endpoints RED metrics. By default `false` which means disabled. If you want to analyze all your request and generate RED metrics for each endpoint of your registered services in the mesh, set it to `true`. - `tier1Cluster` boolean: **DEPRECATED**: This should not be set through Control plane API Instead use TSB Cluster API. Indicates that this cluster is used for tier1 gateways. Tier one clusters can only contain tier 1 gateways. Non-tier1 clusters contain tier2 gateways but not tier 1. - `secrets` V1alpha1Secrets: Secrets available in the ControlPlane installation. - `tsb` SecretsTSB: Secrets to reach the TSB Management Plane. - `cacert` string: CA certificate used to verify TLS certs exposed the Management Plane (front envoy). - `elasticsearch` SecretsElasticSearch: Secrets to reach the Elasticsearch. - `username` string: The username to access Elasticsearch. - `password` string: The password to access Elasticsearch. - `cacert` string: Elasticsearch CA cert TLS used by control plane to verify TLS connection. - `xcp` V1alpha1SecretsXCP: Secrets to reach the XCP Central in the Management Plane. - `autoGenerateCerts` boolean: Enabling this will auto generate XCP Edge certificate if mTLS is enabled to authenticate to XCP Central. Requires cert-manager. - `rootca` string: CA certificate of XCP components. - `rootcakey` string: Key of the CA certificate of XCP components. - `edge` XCPEdge: Secrets for the XCP Edge component. - `cert` string: Edge certificate used for mTLS with XCP Central. - `key` string: Key of the Edge certificate used for mTLS with XCP Central. - `token` string: JWT token used to authenticate XCP Edge against the XCP Central. - `clusterServiceAccount` SecretsClusterServiceAccount: Cluster service account used to authenticate to the Management Plane. - `clusterFQN` string: TSB FQN of the onboarded cluster resource. This will be generate tokens for all Control Plane agents. - `JWK` string: Literal JWK used to generate and sign the tokens for all the Control Plane agents. - `encodedJWK` string: Base64-encoded JWK used to generate and sign the tokens for all the Control Plane agents. - `operator` V1alpha1Operator: Operator values for the TSB operator application. - `deployment` V1alpha1OperatorDeployment: Values for the TSB operator deployment. - `affinity` InstallkubernetesAffinity - `nodeAffinity` InstallkubernetesNodeAffinity - … - `podAffinity` InstallkubernetesPodAffinity - … - `podAntiAffinity` InstallkubernetesPodAntiAffinity - … - `annotations` object - `env` InstallkubernetesEnvVar[] - `name` string - `value` string - `valueFrom` InstallkubernetesEnvVarSource - … - `podAnnotations` object: Custom collection of annotations to add to the pod. - `replicaCount` integer: Number of replicas managed by the deployment. - `strategy` InstallkubernetesDeploymentStrategy - `type` string - `rollingUpdate` InstallkubernetesRollingUpdateDeployment: Mirrors k8s.io.api.apps.v1.RollingUpdateDeployment for unmarshalling. - … - `tolerations` Corev1Toleration[] - `key` string - `operator` string - `value` string - `effect` string - `tolerationSeconds` string, int64 - `service` V1alpha1OperatorService: Values for the TSB operator service. - `annotations` object - `serviceAccount` V1alpha1OperatorServiceAccount: Values for the TSB operator service account. - `annotations` object: Custom collection of annotations to add to the service account. - `imagePullSecrets` string[]: Collection of secrets names required to be able to pull images from the registry. - `pullSecret` string - `pullUsername` string: Used along pull password and the provided image registry to generate a Docker config JSON that will be stored as a pull secret. - `pullPassword` string: Used along pull username and the provided image registry to generate a Docker config JSON that will be stored as a pull secret. - `labels` object - `locality` V2Locality: The region the cluster resides. Used for failover based routing when configured in the workspace or global settings. - `region` string, required: The geographic location of the cluster. - `namespaceScope` V2NamespaceScoping: Configure the default scoping of namespaces in this cluster. - `scope` 'GLOBAL' | 'LOCAL': - GLOBAL: Global configures namespaces in this cluster to be considered global. Namespaces that exist in other clusters with the same name will be considered to be the same logical namespace. - LOCAL: Configures local scoping for namespaces, so that namespaces with the same name in different clusters will not be considered the same logical namespace. - `exceptions` string[]: Namespaces to be excluded form the default scope. If the scope is set to global, this list will contain namespaces that are considered local. If the scope is set to local, this list will contain namespaces that are considered global. - `namespaces` Tsbv2Namespace[] - `name` string - `services` Tsbv2Service[] - `hostname` string: The hostname by which this service is accessed. Can correspond to the hostname of an internal service or that of a virtual host on a gateway. - `namespace` string: namespace associated with the service. - `selector` object: label selectors associated with the service. - `ports` ServicePort[]: The set of ports on which this service is exposed. - `number` integer: A valid non-negative integer port number. - `name` string: Name assigned to the port. - `kubernetesNodePort` integer: Indicates the node port attached to a physical deployment on a kubernetes cluster. - `kubernetesServiceIp` string: For services running on Kubernetes, the cluster IP of the service in the cluster. - `kubernetesExternalAddresses` string[]: For kubernetes services of type load balancer, this field contains the list of lb hostnames or IPs assigned to the service. For services of type nodePort, this field contains the IP addresses of the nodes in the cluster. - `numKubernetesEndpoints` integer: The number of kubernetes pods providing this service. Note that a service could be hosted on both pods and VMs. - `numVmEndpoints` integer: The number of VMs providing this service. - `meshExternal` boolean - `gatewayHost` boolean: Indicates that this is a service representing a gateway host (used for multi-cluster scenarios). - `numHops` integer: The number of hops from the advertising cluster to the ingress gateway that is exposing the gateway host. If the advertiser is the cluster owning the ingress gateway host, the hops is 0. If the advertiser is a tier1 cluster exposing gateways via internal or external Servers, the path hops is 1. - `kubernetesServiceFqdn` string: If this is a gateway host service, this field will indicate the kubernetes service that is running as a gateway and exposing this host to the outside world. - `name` string - `subsets` string[] - `canonicalName` string: 17 to match xcp api and make conversion easier. - `workloads` V2Workload[]: Workloads implementing the Service. - `address` string: Routable address of the workload. - `name` string: Instance name of the workload. - `isVm` boolean: Indicates whether the workload is kubernetes endpoint or vm. - `proxy` WorkloadProxy: Info about proxy attached to a workload. - `controlPlaneAddress` string: Address/service of control plane entity controlling the proxy like istiod.istio-system.svc:15012. - `envoyVersion` string: Envoy version of the proxy. - `istioVersion` string: Istio version of the proxy. - `status` object - `tier1GatewayHost` boolean: Indicates that this is a service representing a gateway host acting as a tier1 gateway. This field is being introduced specifically for the hostnames exposed through Tier1Gateway's ExternalServers. Fo all other hostnames that are exposed exposed on IngessGateways or Tier1Gateway's InternalServers gateway_host field is set to true because that is used for multicluster routing setup. For ExternalServer hostnames multicluster cluster routing is not setup and hence UI on finding gateway_host as false shows these hostnames as internal services incorrectly. To help UI, this field is being introduced. UI would look for either gateway_host or tier1_gateway_host to decide if the hostname is a gateway hostname. - `spiffeIds` string[]: List of SPIFFE identities used by the workloads of the service. - `network` string: The network (e.g., VPC) where this cluster is present. All clusters within the same network will be assumed to be reachable for the purposes of multi-cluster routing. In addition, networks marked as reachable from one another in SystemSettings will also be used for multi-cluster routing. - `serviceAccount` Tsbv2ServiceAccount: `ServiceAccount` represents a service account that can be used to access the TSB platform. Service accounts have a set of associated public and private keys that can be used to generate signed JWT tokens that are suitable to authenticate to TSB. A default key-pair is generated on service account creation and the public key is stored in TSB. Private keys are returned when service accounts are created, but TSB will not store them. It is up to the client to store them securely. - `fqn` string - `displayName` string - `etag` string - `description` string: A description of the resource. - `keys` ServiceAccountKeyPair[] - `id` string - `publicKey` string - `privateKey` string - `encoding` 'PEM' | 'JWK' - `defaultToken` string - `state` ClusterState - `lastSyncTime` string, date-time - `provider` string - `istioVersions` string[]: This shows currently running istio versions in the cluster. - `xcpVersion` string - `tsbCpVersion` string - `discoveredLocality` V2Locality: The region the cluster resides. Used for failover based routing when configured in the workspace or global settings. - `region` string, required: The geographic location of the cluster. - `tier1Cluster` boolean: Indicates whether this cluster is hosting a tier1 gateway or not. Tier1 clusters cannot host other gateways or workloads. Defaults to false if not specified. - `tokenTtl` string: Lifetime of the tokens. Defaults to 1hr. - `trustDomain` string: Trust domain for this cluster, used for multi-cluster routing. It must be unique for every cluster and should match the one configured in the local control plane. This value is optional, and will be updated by the local control plane agents. However, it is recommended to set it, if known, so that multi-cluster routing works without having to wait for the local control planes to update it. ## Response `200` A successful response. - V2Cluster: A Kubernetes cluster managing both pods and VMs. - `fqn` string - `displayName` string - `etag` string - `description` string - `tokenTtl` string: Lifetime of the tokens. Defaults to 1hr. - `network` string: The network (e.g., VPC) where this cluster is present. All clusters within the same network will be assumed to be reachable for the purposes of multi-cluster routing. In addition, networks marked as reachable from one another in SystemSettings will also be used for multi-cluster routing. - `tier1Cluster` boolean: Indicates whether this cluster is hosting a tier1 gateway or not. Tier1 clusters cannot host other gateways or workloads. Defaults to false if not specified. - `namespaces` Tsbv2Namespace[] - `name` string - `services` Tsbv2Service[] - `hostname` string: The hostname by which this service is accessed. Can correspond to the hostname of an internal service or that of a virtual host on a gateway. - `namespace` string: namespace associated with the service. - `selector` object: label selectors associated with the service. - `ports` ServicePort[]: The set of ports on which this service is exposed. - `number` integer: A valid non-negative integer port number. - `name` string: Name assigned to the port. - `kubernetesNodePort` integer: Indicates the node port attached to a physical deployment on a kubernetes cluster. - `kubernetesServiceIp` string: For services running on Kubernetes, the cluster IP of the service in the cluster. - `kubernetesExternalAddresses` string[]: For kubernetes services of type load balancer, this field contains the list of lb hostnames or IPs assigned to the service. For services of type nodePort, this field contains the IP addresses of the nodes in the cluster. - `numKubernetesEndpoints` integer: The number of kubernetes pods providing this service. Note that a service could be hosted on both pods and VMs. - `numVmEndpoints` integer: The number of VMs providing this service. - `meshExternal` boolean - `gatewayHost` boolean: Indicates that this is a service representing a gateway host (used for multi-cluster scenarios). - `numHops` integer: The number of hops from the advertising cluster to the ingress gateway that is exposing the gateway host. If the advertiser is the cluster owning the ingress gateway host, the hops is 0. If the advertiser is a tier1 cluster exposing gateways via internal or external Servers, the path hops is 1. - `kubernetesServiceFqdn` string: If this is a gateway host service, this field will indicate the kubernetes service that is running as a gateway and exposing this host to the outside world. - `name` string - `subsets` string[] - `canonicalName` string: 17 to match xcp api and make conversion easier. - `workloads` V2Workload[]: Workloads implementing the Service. - `address` string: Routable address of the workload. - `name` string: Instance name of the workload. - `isVm` boolean: Indicates whether the workload is kubernetes endpoint or vm. - `proxy` WorkloadProxy: Info about proxy attached to a workload. - `controlPlaneAddress` string: Address/service of control plane entity controlling the proxy like istiod.istio-system.svc:15012. - `envoyVersion` string: Envoy version of the proxy. - `istioVersion` string: Istio version of the proxy. - `status` object - `tier1GatewayHost` boolean: Indicates that this is a service representing a gateway host acting as a tier1 gateway. This field is being introduced specifically for the hostnames exposed through Tier1Gateway's ExternalServers. Fo all other hostnames that are exposed exposed on IngessGateways or Tier1Gateway's InternalServers gateway_host field is set to true because that is used for multicluster routing setup. For ExternalServer hostnames multicluster cluster routing is not setup and hence UI on finding gateway_host as false shows these hostnames as internal services incorrectly. To help UI, this field is being introduced. UI would look for either gateway_host or tier1_gateway_host to decide if the hostname is a gateway hostname. - `spiffeIds` string[]: List of SPIFFE identities used by the workloads of the service. - `labels` object - `locality` V2Locality: The region the cluster resides. Used for failover based routing when configured in the workspace or global settings. - `region` string, required: The geographic location of the cluster. - `trustDomain` string: Trust domain for this cluster, used for multi-cluster routing. It must be unique for every cluster and should match the one configured in the local control plane. This value is optional, and will be updated by the local control plane agents. However, it is recommended to set it, if known, so that multi-cluster routing works without having to wait for the local control planes to update it. - `namespaceScope` V2NamespaceScoping: Configure the default scoping of namespaces in this cluster. - `scope` 'GLOBAL' | 'LOCAL': - GLOBAL: Global configures namespaces in this cluster to be considered global. Namespaces that exist in other clusters with the same name will be considered to be the same logical namespace. - LOCAL: Configures local scoping for namespaces, so that namespaces with the same name in different clusters will not be considered the same logical namespace. - `exceptions` string[]: Namespaces to be excluded form the default scope. If the scope is set to global, this list will contain namespaces that are considered local. If the scope is set to local, this list will contain namespaces that are considered global. - `state` ClusterState - `lastSyncTime` string, date-time - `provider` string - `istioVersions` string[]: This shows currently running istio versions in the cluster. - `xcpVersion` string - `tsbCpVersion` string - `discoveredLocality` V2Locality: The region the cluster resides. Used for failover based routing when configured in the workspace or global settings. - `region` string, required: The geographic location of the cluster. - `serviceAccount` Tsbv2ServiceAccount: `ServiceAccount` represents a service account that can be used to access the TSB platform. Service accounts have a set of associated public and private keys that can be used to generate signed JWT tokens that are suitable to authenticate to TSB. A default key-pair is generated on service account creation and the public key is stored in TSB. Private keys are returned when service accounts are created, but TSB will not store them. It is up to the client to store them securely. - `fqn` string - `displayName` string - `etag` string - `description` string: A description of the resource. - `keys` ServiceAccountKeyPair[] - `id` string - `publicKey` string - `privateKey` string - `encoding` 'PEM' | 'JWK' - `defaultToken` string - `installTemplate` ClusterInstallTemplate: InstallTemplate provides templates ready to be used in the ControlPlane (cluster onboard) installation. - `message` string - `helm` V1alpha1Values: Values available for the TSB Control Plane chart. This is an alpha API, so future versions could include breaking changes. - `image` Commonv1alpha1Image: Values for the TSB operator image. - `registry` string: Registry used to download the operator image. - `tag` string: The tag of the operator image. - `spec` V1alpha1ControlPlaneSpec: ControlPlaneSpec defines the desired installed state of control plane components. Specifying a minimal ControlPlaneSpec with hub, clusterName, and managementPlane set will create an installation with sensible defaults. - `hub` string, required: TSB container hub path e.g. docker.io/tetrate. - `imagePullSecrets` InstallkubernetesLocalObjectReference[]: Pull secrets can be specified globally for all components, or defined into the `kubeSpec.serviceAccount` of every component if needed. In case both are defined, the most specific one (the one defined at the component) level is used. List of references to secrets in the same namespace to use for pulling any images in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet. More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#service_account-v1-core - `name` string: Name of the referent. - `components` V1alpha1ControlPlaneComponentSet: The set of components that make up the control plane. Use this to override application settings or Kubernetes settings for each individual component. - `collector` V1alpha1OpenTelemetryCollector: Kubernetes settings for the OpenTelemetryCollector component. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevel` string: Specifies the log level for OTEL collector component. Supported log level: "debug", "info", "warn", "error", "dpanic", "panic", and "fatal". - `oap` V1alpha1Oap: Kubernetes settings for the OAP (SkyWalking) component. - `streamingLogEnabled` boolean: Feature flag to determine whether on-demand streaming logs should be enabled. - `onDemandEnvoyMetricsEnabled` boolean: Feature flag to determine whether on-demand envoy metrics should be enabled. If enabled, the envoy proxy will provide a set of metrics that can be queried using the metrics service. OAP will provide a query API that can be used to collect envoy proxy metrics for specific pods. This is only for temporary and real-time queries that can be used, for example, for application troubleshooting use cases. These metrics are not persisted. - `storageIndexMergingEnabled` boolean: Feature flag to determine whether metrics/meter and records should be shard into multi-physical indices, or instead if they should be merged into a single physical index. By default "false", metric/meter and records are sharded into multi-physical indices. Instead of sharding, if enabled by setting it to "true", metrics/meter and records will be merged into one physical index template `metrics-all` and `records-all`. This feature flag must be set on all clusters and have the same value as the management plane's one, otherwise control plane observability data could be written to the wrong or not existing index. In this storage mode, user can adjust each concrete index should have to scale out by setting `storageSpecificIndexSettings` field in the management plane install manifest. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevel` string: Specifies the log level for OAP component. Supported log level: "all", "debug", "info", "warn", "error", "fatal", "off" and "trace". - `xcp` Controlplanev1alpha1XCP: Kubernetes settings for the XCP component. - `centralAuthMode` 'UNKNOWN' | 'MUTUAL_TLS' | 'JWT': - UNKNOWN: Default when unset, do not use - MUTUAL_TLS: GRPC stream is encrypted with mutual TLS - JWT: XCP Edges present a JWT bearer token in the GRPC headers - `configProtection` CommonConfigProtection - … - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `revision` string: $hide_from_docs Configures the istio revision tag. If configured, istio upgrade will not be in-place upgrade. A new istio control plane with the configured revision will be deployed. Selectively sidecars and gateways could be moved to newer control plane. Note that it is not the istio version. Istio version is fixed for a particular tsb version and that is not a configurable setting. Revision should be configured to human readable value for example tsb-1-5. For further reference, https://istio.io/latest/blog/2020/multiple-control-planes/#configuring Deprecated: replaced by isolation_boundaries, where each isolation boundary can have mulitple revisions. - `isolationBoundaries` V1alpha1IsolationBoundary[]: Configures Isolated Istio environments along with Istio revisions for each environment. IsolationBoundaries can be empty when the feature flag IstioIsolationBoundaries is disabled. Once enabled, isolation boundaries can be configured. - … - `enableHttpMeshInternalIdentityPropagation` boolean: Enables HTTP mesh internal service identity propagation across gateway hops, utilizing the propagated identity for evaluating TSB RBAC rules. Users should enable this feature when they want to create RBAC rules around request's origin client identity for east west traffic. The most common case for this would be when using authorization features such as ALLOW/DENY rules mode and ServiceSecuritySettings in cross-cluster environment. This feature is disabled by default. - `centralProvidedCaCert` boolean: If true, obtain the CA cert for Istio from XCP central. To enable it, the XCP Central needs to be configured with `certIssuer.clusterIntermediateCASettings: {}`. - `logLevels` object: Loglevel for XCP. Supported log level: "none", "fatal", "error", "warn", "info", "debug". - `istio` V1alpha1Istio: Mesh and Kubernetes settings for Istio. - `tsbVersion` string: Specifies the tsb release version. This is used by the tsb control plane operator in determining the xcp version, which would eventually decide Istio version. If not provided explicitly, this defaults to the current tsb version. - `mountInternalWasmExtensions` boolean: When this flag is set, the TSB internal WASM extensions will be mounted into the Sidecar, Ingress and Egress gateway pods automatically. These extensions will be loaded as local files instead of being downloaded from a remote OCI registry or HTTP endpoint. This is currently disabled by default, but in future releases this flag will be enabled by default. - `kubeSpec` KubernetesKubernetesIstioComponentSpec: KubernetesIstioComponentSpec is the common set of Kubernetes resource configuration for Istio. It differs from the standard component specs in that it supports CNI configuration. - … - `traceSamplingRate` number, double: The percentage of traces Envoy will sample. - `defaultWorkloadCertTTL` string: The default TTL of issued workload certificates. This sets both the default client-side CSR TTL and the default server-side issued certificate TTL. - `maxWorkloadCertTTL` string: The maximum TTL that can be set in issued workload certificates. - `trustDomain` string: The trust domain corresponds to the trust root of a system. Refer to [SPIFFE-ID](https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE-ID.md#21-trust-domain). If omitted, TSB will configure the trust domain as `CLUSTER_NAME.tsb.local`, where `CLUSTER_NAME` is the name of the cluster object in TSB for this control plane. - `baseOverlays` V1alpha1K8sObjectOverlay[]: The overlays applied to the Istio base component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in `kubeSpec.overlays` are ignored. - … - `pilotOverlays` V1alpha1K8sObjectOverlay[]: The overlays applied to the Istio pilot component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in `kubeSpec.overlays` are ignored. - … - `cniOverlays` V1alpha1K8sObjectOverlay[]: The overlays applied to the Istio CNI component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in `kubeSpec.overlays` are ignored. - … - `logLevels` object: Specifies the global logging level settings for the Istio control plane components. - `rateLimitServer` V1alpha1RateLimitServer - `backend` RateLimitServerBackend, required: External Backend Database types. This points to the backend used by the ratelimit server as a key/value store. - … - `domain` string: The domain field allows ratelimits to be namespaced to a certain domain. To support common ratelimits across multiple clusters set this string to a common value, across them. This assumes that the same backend (uri) is being used. By default the domain is set to the name of the control plane cluster. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `hpaAdapter` V1alpha1HpaAdapter: Kubernetes settings for the OAP (SkyWalking) HPA adapter component. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `onboarding` V1alpha1Onboarding: Settings for the `Workload Onboarding` component. - `operator` V1alpha1OnboardingOperator: Kubernetes settings for the `Workload Onboarding Operator` component. - … - `repository` V1alpha1OnboardingRepository: Kubernetes settings for the `Workload Onboarding Repository` component. - … - `plane` Controlplanev1alpha1OnboardingPlane: Configure `Workload Onboarding Plane` component. - … - `satellite` V1alpha1Satellite: Kubernetes settings for the Satellite (SkyWalking-Satellite) component. - `enabled` boolean: Satellite is an optional component. If enabled is false, this component will not be installed. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevel` string: Specifies the log level for the component. Supported log level: "panic", "fatal", "info", "warn", "error", "debug" and "trace". - `ngac` V1alpha1NGAC: Kubernetes settings for the NGAC component. - `enabled` boolean: NGAC is an experimental component. If enabled is false, this component will not be installed. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevels` object: The log level configuration by scopes. Supported log level: "none", "error", "info", "debug". - `gitops` CommonGitOps: The GitOps component configures the features that allow integrating the Management Plane and/or the Control Plane cluster with Continuous Deployment pipelines. - `enabled` boolean: The GitOps component is in beta and disabled by default. If Management and Control Planes are installed in the same cluster, Continuous Deployment Integration should only be enabled in one of both planes. However, if the GitOps component is enabled in both planes, only the Control Plane GitOps component will remain enabled. The Management Plane GitOps component will not be enabled, even though it is explicitly enabled. - `reconcileInterval` string: Interval at which the reconcile process will run. The reconcile process will read all TSB CRs that exist in the cluster and reapply them to the management plane, to make sure the cluster CRs remain as the source of truth. Format: 1h/1m/1s/1ms. A value of 0 disables the reconcile loop. Default: 10m. - `batchWindow` string: When configured, all admission requests will be paused for the configured duration. Once the window interval is closed, all paused admission requests will be sent together to the Management Plane as a single request. Batching of requests is disabled by default and should be enabled only if there is high concurrency and ordering of resources could be an issue. By configuring a batch window the concurrency and ordering issues may be mitigated, although it will introduce a constant latency to all requests of the configured time window. When enabled, it is recommended to use a small value, for example 1 second. - `managementplaneRequestTimeout` string: The GitOps component performs operations against the management plane through the k8s webhook. This allows configuring the duration of each operation in order to fail early if it takes too much. This value cannot be lower than `webhook_timeout` due to the request being tied to the ones received by the k8s webhook. Format: 1h/1m/1s/1m. Any value <= 0 will be reset to the default value. Default: 25s. - `reconcileRequestTimeout` string: The GitOps component performs operations against the management plane internal reconcile loop. This allows configuring the duration of each operation to fail early if it takes too long. Format: 1h/1m/1s/1m. Any value <= 0 will be reset to the default value. Default: 2m. - `webhookTimeout` string: Timeout that will be set in the k8s gitops webhook resource. Format: 1h/1m/1s/1m. Default: 30s. Allowed values must be between 0s and 30s. - `internalCertProvider` CommonInternalCertProvider: InternalCertProvider describes the certificate provider configuration for TSB internal purposes like kubernetes webhook certificate. TSB supports cert-manager out of the box. - `certManager` CommonCertManagerSettings: CertManagerSettings represents the settings used for the cert-manager installation. TSB supports installing and managing the lifecycle of the cert-manager installation. - … - `custom` CommonCustomCertProviderSettings: CustomCertProviderSettings represents the settings used for the custom certificate provider. Users can configure the CSR signer required for certificate signing and point to the CA bundle to be used to validate the certificates. - … - `defaultKubeSpec` KubernetesKubernetesSpec: KubernetesSpec is a common set of Kubernetes resource configuration for the install CRs, that will be common to all of its components. - `deployment` KubernetesGlobalDeployment - … - `service` KubernetesGlobalService - … - `account` InstallkubernetesServiceAccount - … - `job` KubernetesGlobalJob - … - `wasmfetcher` V1alpha1WASMFetcher: Settings for the WASM Fetcher component. - `cacheDisableInsecureRegistries` boolean: Denies insecure registries to be used for fetching WASM modules. Defaults to `false`. - `cacheExpiration` string: WASM Module cache expiration time. Defaults to `24h`. - `cacheMaxRetries` integer: Maximum number of retries when fetching WASM modules from the OCI registry. Defaults to `5`. - `cachePurgeInterval` string: WASM cache purge interval to periodically clean up the stale WASM modules. Defaults to `1h`. - `cacheRequestTimeout` string: Specifies the timeout used when retrieving the WASM plugin from the OCI registry. Defaults to `15s`. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `logLevels` object: The log level configuration by scopes. Supported log levels: "none", "error", "info", "debug". - `defaultLogLevel` string: The default log level for all components if the per component log level config is not specified. Note that the supported log level for different components can be different. - `route53Controller` V1alpha1Route53Controller: Kubernetes settings for the Route53 Integration Controller component. - `kubeSpec` KubernetesKubernetesComponentSpec: KubernetesComponentSpec is a common set of Kubernetes resource configuration for components. - … - `providerSettings` V1alpha1ProviderSettings: Configure Kubernetes provider specific settings. For example to configure EKS to use network load balancers (NLB) by default: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: providerSettings: eks: useNlbByDefault: true ``` To configure Route53 the only option that you must specify is the Service Account name to use for IAM role. You should create the Service Account before enabling the Route53 integration controller. You can do that using `eksctl`. Example: ```bash SA_NAME=route53-controller CP_NAMESPACE=istio-system eksctl create iamserviceaccount \ --cluster $EKS_CLUSTER_NAME \ --name $SA_NAME \ --namespace $CP_NAMESPACE \ --attach-policy-arn $POLICY_ARN \ --approve ``` where: * $EKS_CLUSTER_NAME is the name of the EKS cluster. * $SA_NAME is the name of the Service Account to create. * $CP_NAMESPACE is the namespace where the Control Plane is installed. Usually istio-system. * $POLICY_ARN is the ARN of the policy to attach to the Service Account - the policy should allow the Service Account to manage Route53 resources. More details can be found in the [Publishing a Service docs](https://docs-preview.tetrate.io/service-express/Tech-Preview/getting-started/publish-service) After creating the Service Account you can enable the Route53 integration controller using the following configuration: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: providerSettings: route53: serviceAccountName: $SA_NAME ``` - `eks` V1alpha1EKSSettings: Settings specific to Elastic Kubernetes Service (EKS). - `useNlbByDefault` boolean: When true, gateways will be configured to use NLBs with cross zone load balancing enabled when the load balancer type is not configured. When false, no additional annotations will be added. - `route53` V1alpha1Route53Settings: Settings for integration with Route53 service. - `serviceAccountName` string, required: Service account name to use for IAM role. Required. - `namespaceSelector` V1alpha1Route53SettingsNamespaceSelector: NamespaceSelector specifies which namespaces controller will watch. - … - `policy` 'SYNC' | 'UPSERT_ONLY' | 'CREATE_ONLY': Policy that defines how DNS records are managed. - SYNC: Allow full synchronization. - UPSERT_ONLY: Don't allow delete DNS records. - CREATE_ONLY: Allow only creating DNS records. - `domainFilter` string[]: Limit possible target zones by a domain suffix; specify multiple times for multiple domains. Optional. - `interval` string: Duration of interval between individual synchronizations. Default: 60s. - `ttl` string, int64: Default TTL (in seconds) value for DNS records. Default: 300. - `evaluateTargetHealth` boolean: Control whether to evaluate the health of a DNS target. Default: true. - `filterSettings` Route53SettingsFilterSettings: Filter settings for route53 controller. - … - `managementPlane` V1alpha1ManagementPlaneSettings, required: Configure the management plane connection. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: managementPlane: host: tsb.tetrate.io port: 8443 selfSigned: true clusterName: control-plane-cluster ``` - `host` string, required: Management plane host address (can be hostname or IPv4/IPv6 address). - `port` integer, required: Port management plane is listening on. - `selfSigned` boolean: Management plane uses a self signed or private TLS certificate. If true, the CA bundle used to verify the MP's TLS certificate must be in a secret `mp-certs` under the key `ca.crt`. - `clusterName` string, required: The name of the Cluster object that was created in the Management Plane representing this Control Plane cluster. - `meshExpansion` V1alpha1MeshExpansionSettings: Configure mesh expansion to connect workloads external to Kubernetes to the mesh. To enable mesh expansion set it to an empty object: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: {} ``` If external workloads are unable to communicate with the default mesh expansion gateway via external IPs or hostnames, then you must specify the gateway that enables them to do so. This custom gateway must be configured to forward this communication to the VM gateway service: ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: customGateway: host: customgateway.tetrate.io port: 15443 ``` To automate onboarding of workloads from auto-scaling groups of VMs, you need to enable the `Workload Onboarding Plane`. `Workload Onboarding Agent`, a component that you install next to the workload, will connect to the `Workload Onboarding Plane` to authenticate itself, ask permission to join the mesh, register the workload into the mesh and retrieve boot configuration required to start `Istio Sidecar`. All communication between the `Workload Onboarding Agent` and the `Workload Onboarding Plane` must occur over TLS. Therefore, to enable `Workload Onboarding Plane` you must provide a TLS certificate for the endpoint that exposes `Workload Onboarding API` to `Workload Onboarding Agents`. Make sure that TLS certificate is signed by the certificate authority known to `Workload Onboarding Agents`. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: onboarding: endpoint: hosts: - onboarding.example.org secretName: onboarding-tls-cert tokenIssuer: jwt: expiration: 1h localRepository: {} ``` To onboard workloads from custom on-premise environments, you can leverage support for [OIDC ID Tokens](https://openid.net/specs/openid-connect-core-1_0.html#IDToken). If workloads in your custom environment can authenticate themselves by means of an [OIDC ID Token](https://openid.net/specs/openid-connect-core-1_0.html#IDToken), you can define a list of JWT issuers permitted by the `Workload Onboarding Plane`. For example, ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: onboarding: endpoint: hosts: - onboarding.example.org secretName: onboarding-tls-cert localRepository: {} workloads: authentication: jwt: issuers: - issuer: "https://mycompany.corp" jwksUri: "https://mycompany.corp/jwks.json" shortName: "mycorp" tokenFields: attributes: jsonPath: .custom_attributes ``` To ensure there will be no traffic loss when an onboarded workload gets shutdown, you can configure the time period to delay the shutdown for after deregistering the workload from the mesh, which will give enough time to reconfigure all affected mesh nodes to not load balance requests to the deregistered workload before it becomes unavailable. For example, ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshExpansion: onboarding: endpoint: hosts: - onboarding.example.org secretName: onboarding-tls-cert localRepository: {} workloads: deregistration: propagationDelay: 15s ``` - `customGateway` MeshExpansionSettingsGateway: A custom mesh expansion gateway. This is required when the workload can't access the default gateway directly via the external IP or hostname. - `host` string, required: Mesh expansion gateway host address (can be hostname or IP address). - `port` integer, required: Port mesh expansion gateway is listening on. - `onboarding` V1alpha1MeshExpansionSettingsOnboardingPlane: Configuration of the `Workload Onboarding Plane`. - `uid` string: Unique identifier of this particular installation of the `Workload Onboarding Plane`. Is used in the workload authentication flow to prevent replay attacks that abuse compromised workload credentials intended for a different installation of the `Workload Onboarding Plane`. Defaults to an auto-generated UUID. - `endpoint` OnboardingPlaneEndpoint, required: Configuration of the endpoint exposing `Workload Onboarding API` to `Workload Onboarding Agents`. - … - `tokenIssuer` OnboardingPlaneTokenIssuer: Configuration of the built-in `Workload Onboarding Token Issuer`. - … - `localRepository` OnboardingPlaneLocalRepository: Configuration of the local repository with `DEB` and `RPM` packages of the `Workload Onboarding Agent` and `Istio Sidecar`. - `workloads` V1alpha1WorkloadConfiguration: WorkloadConfiguration specifies configuration of the workload handling. - … - `telemetryStore` ControlPlaneSpecTelemetryStore, required: Configure the store that TSB will use to persist application telemetry data. Select one of the `TelemetryStore` settings to see complete examples. - `elastic` V1alpha1ElasticSearchSettings: Configure an Elasticsearch connection. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: telemetryStore: elastic: host: elastic port: 5678 protocol: https selfSigned: true version: 7 ``` - `host` string, required: Elasticsearch host address (can be hostname or IP address). - `port` integer, required: Port Elasticsearch is listening on. - `protocol` 'https' | 'http': The list of supported protocols to communicate with Elasticsearch. - `selfSigned` boolean: Use Self-Signed certificates. The Self-signed CA bundle and key must be in a secret called es-certs. - `version` integer: DEPRECATED: Major version of the Elasticsearch cluster. Currently supported Elasticsearch major versions are `6`, `7`, and `8`. - `meshObservability` ControlPlaneSpecMeshObservability: Configure how the mesh should be observed, which observability functionalities should be enabled to observe your registered services in the mesh, and the store properties that TSB will use to persist application observability data like metrics, traces, logs. If omitted, the operator will assume a demo installation and for your convenience install a demo grade mesh observability setting. Select one of the `MeshObservability` settings to see complete examples. - `demoSettings` CommonMeshObservabilitySettings: Configure mesh observability. The following examples enable the analysis and generation of RED metrics for each endpoint of your registered services. Notice that both, ManagementPlane and ControlPlane, need to be aligned with this configuration. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ManagementPlane metadata: name: managementplane spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` - `apiEndpointMetricsEnabled` boolean: Toggle to process, analyze, and generate api endpoints RED metrics. By default `false` which means disabled. If you want to analyze all your request and generate RED metrics for each endpoint of your registered services in the mesh, set it to `true`. - `settings` CommonMeshObservabilitySettings: Configure mesh observability. The following examples enable the analysis and generation of RED metrics for each endpoint of your registered services. Notice that both, ManagementPlane and ControlPlane, need to be aligned with this configuration. ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ManagementPlane metadata: name: managementplane spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` ```yaml apiVersion: install.tetrate.io/v1alpha1 kind: ControlPlane metadata: name: controlplane namespace: istio-system spec: meshObservability: settings: apiEndpointMetricsEnabled: true ``` - `apiEndpointMetricsEnabled` boolean: Toggle to process, analyze, and generate api endpoints RED metrics. By default `false` which means disabled. If you want to analyze all your request and generate RED metrics for each endpoint of your registered services in the mesh, set it to `true`. - `tier1Cluster` boolean: **DEPRECATED**: This should not be set through Control plane API Instead use TSB Cluster API. Indicates that this cluster is used for tier1 gateways. Tier one clusters can only contain tier 1 gateways. Non-tier1 clusters contain tier2 gateways but not tier 1. - `secrets` V1alpha1Secrets: Secrets available in the ControlPlane installation. - `tsb` SecretsTSB: Secrets to reach the TSB Management Plane. - `cacert` string: CA certificate used to verify TLS certs exposed the Management Plane (front envoy). - `elasticsearch` SecretsElasticSearch: Secrets to reach the Elasticsearch. - `username` string: The username to access Elasticsearch. - `password` string: The password to access Elasticsearch. - `cacert` string: Elasticsearch CA cert TLS used by control plane to verify TLS connection. - `xcp` V1alpha1SecretsXCP: Secrets to reach the XCP Central in the Management Plane. - `autoGenerateCerts` boolean: Enabling this will auto generate XCP Edge certificate if mTLS is enabled to authenticate to XCP Central. Requires cert-manager. - `rootca` string: CA certificate of XCP components. - `rootcakey` string: Key of the CA certificate of XCP components. - `edge` XCPEdge: Secrets for the XCP Edge component. - `cert` string: Edge certificate used for mTLS with XCP Central. - `key` string: Key of the Edge certificate used for mTLS with XCP Central. - `token` string: JWT token used to authenticate XCP Edge against the XCP Central. - `clusterServiceAccount` SecretsClusterServiceAccount: Cluster service account used to authenticate to the Management Plane. - `clusterFQN` string: TSB FQN of the onboarded cluster resource. This will be generate tokens for all Control Plane agents. - `JWK` string: Literal JWK used to generate and sign the tokens for all the Control Plane agents. - `encodedJWK` string: Base64-encoded JWK used to generate and sign the tokens for all the Control Plane agents. - `operator` V1alpha1Operator: Operator values for the TSB operator application. - `deployment` V1alpha1OperatorDeployment: Values for the TSB operator deployment. - `affinity` InstallkubernetesAffinity - `nodeAffinity` InstallkubernetesNodeAffinity - … - `podAffinity` InstallkubernetesPodAffinity - … - `podAntiAffinity` InstallkubernetesPodAntiAffinity - … - `annotations` object - `env` InstallkubernetesEnvVar[] - `name` string - `value` string - `valueFrom` InstallkubernetesEnvVarSource - … - `podAnnotations` object: Custom collection of annotations to add to the pod. - `replicaCount` integer: Number of replicas managed by the deployment. - `strategy` InstallkubernetesDeploymentStrategy - `type` string - `rollingUpdate` InstallkubernetesRollingUpdateDeployment: Mirrors k8s.io.api.apps.v1.RollingUpdateDeployment for unmarshalling. - … - `tolerations` Corev1Toleration[] - `key` string - `operator` string - `value` string - `effect` string - `tolerationSeconds` string, int64 - `service` V1alpha1OperatorService: Values for the TSB operator service. - `annotations` object - `serviceAccount` V1alpha1OperatorServiceAccount: Values for the TSB operator service account. - `annotations` object: Custom collection of annotations to add to the service account. - `imagePullSecrets` string[]: Collection of secrets names required to be able to pull images from the registry. - `pullSecret` string - `pullUsername` string: Used along pull password and the provided image registry to generate a Docker config JSON that will be stored as a pull secret. - `pullPassword` string: Used along pull username and the provided image registry to generate a Docker config JSON that will be stored as a pull secret. ## Other responses - `default`: An unexpected error response. --- [API](https://skmtc.dev/tetrate/apis/tetrate-service-bridge-api.md) - [All operations](https://skmtc.dev/tetrate/apis/tetrate-service-bridge-api/llms.txt) - [OpenAPI document](https://skmtc.dev/tetrate/apis/tetrate-service-bridge-api/revisions/2f19c3a1ae57?raw)