Logo of Trend Micro
Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Enable GKE Cluster Node Encryption with Customer-Managed Encryption Keys

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: High (not acceptable risk)
Rule ID: GKE-001

Encrypt data on your GKE cluster nodes with Customer-Managed Encryption Keys (CMEKs) for granular control over encryption and decryption. Use Cloud Key Management Service (Cloud KMS) to create and manage your Customer-Managed Encryption Keys. Cloud KMS provides secure and efficient key management, controlled key rotation, and revocation.

This rule resolution is part of the Conformity Security & Compliance tool for GCP.

Security

By default, Google Kubernetes Engine (GKE) encrypts all data at rest using Google-Managed Encryption Keys and manages the encryption/decryption process without user action required. However, if you need to achieve strict compliance or regulatory requirements, you can choose to fully control and manage GKE cluster data encryption yourself, using your own Customer-Managed Encryption Keys (CMEKs). If you have to encrypt business-critical, sensitive, or confidential GKE data, it is strongly recommended to encrypt your GKE cluster nodes using Customer-Managed Encryption Keys.

Audit

Case A: To determine if your GKE Standard cluster nodes are encrypted with Customer-Managed Encryption Keys (CMEKs), perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to examine from the console top navigation bar.

03 Navigate to Kubernetes Engine console available at https://console.cloud.google.com/kubernetes.

04 In the left navigation panel, under Resource Management, choose Clusters and select the OVERVIEW tab to access the list of GKE Standard clusters provisioned for the selected GCP project. A GKE Standard cluster has Mode set to Standard.

05 Click on the name (link) of the GKE Standard cluster that you want to examine.

06 Select the NODES tab to access the node pools created for the selected cluster.

07 Click on the name (link) of the GKE cluster node pool that you want to examine.

08 In the Nodes section, inspect the Boot disk encryption attribute value. If the Boot disk encryption value is set to Google-managed, the nodes deployed for the selected GKE Standard cluster node pool are encrypted with a Google-Managed Encryption Key instead of a Customer-Managed Encryption Key (CMEK).

09 Repeat steps no. 7 and 8 for each node pool deployed for the selected GKE cluster.

10 Repeat steps no. 5 – 9 for each GKE cluster provisioned within the selected GCP project.

11 Repeat steps no. 2 – 10 for each GCP project deployed in your Google Cloud account.

Using GCP CLI

01 Run projects list command (Windows/macOS/Linux) with custom output filters to list the ID of each GCP project available in your Google Cloud account: 

gcloud projects list
	--format="table(projectId)"

02 The command output should return the requested GCP project IDS: 

PROJECT_ID
cc-web-project-123123
cc-dev-project-112233

03 Run container clusters list command (Windows/macOS/Linux) with the ID of the GCP project that you want to examine as the identifier parameter and custom output filters to describe the name and the region of each GKE Standard cluster provisioned for the selected project: 

gcloud container clusters list
	--project cc-web-project-123123
	--format="table(NAME,ZONE,AUTOPILOT)"

04 The command output should return the requested GKE Standard cluster identifiers. A GKE Standard cluster has AUTOPILOT set to {}: 

NAME: cc-gke-backend-cluster
ZONE: us-central1
AUTOPILOT: {}

NAME: cc-gke-frontend-cluster
ZONE: us-central1
AUTOPILOT: {}

05 Run container node-pools list command (Windows/macOS/Linux) with the name of the GKE Standard cluster that you want to examine as the identifier parameter, to describe the name of each node pool provisioned for the selected cluster: 

gcloud container node-pools list
	--cluster=cc-gke-backend-cluster
	--region=us-central1
	--format="(NAME)"

06 The command output should return the requested GKE node pool names: 

NAME:
cc-gke-backend-pool-001
cc-gke-backend-pool-002
cc-gke-backend-pool-003

07 Run container node-pools describe command (Windows/macOS/Linux) with the name of the GKE cluster node pool that you want to examine as the identifier parameter and custom output filters to describe the ID of the Customer-Managed Encryption Key (CMEK) used to encrypt the GKE node data: 

gcloud container node-pools describe cc-gke-backend-pool-001
	--cluster=cc-gke-backend-cluster
	--region=us-central1
	--format="json(config.bootDiskKmsKey)"

08 The command output should return the full resource ID of the CMEK used to encrypt the GKE node data: 

null

If the container node-pools describe command output returns null, as shown in the example above, the nodes provisioned for the selected GKE Standard cluster node pool are encrypted with a Google-Managed Encryption Key instead of a Customer-Managed Encryption Key (CMEK).

09 Repeat steps no. 7 and 8 for each node pool provisioned for the selected GKE cluster.

10 Repeat steps no. 5 – 9 for each GKE cluster created for the selected GCP project.

11 Repeat steps no. 3 – 10 for each GCP project deployed in your Google Cloud account.

Audit

Case B: To determine if your GKE Autopilot cluster nodes are encrypted with Customer-Managed Encryption Keys (CMEKs), perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to examine from the console top navigation bar.

03 Navigate to Kubernetes Engine console available at https://console.cloud.google.com/kubernetes.

04 In the left navigation panel, under Resource Management, choose Clusters and select the OVERVIEW tab to access the list of GKE Autopilot clusters provisioned for the selected GCP project. A GKE Autopilot cluster has Mode set to Autopilot.

05 Click on the name (link) of the GKE Autopilot cluster that you want to examine.

01 Select the DETAILS tab to view the configuration information available for the selected cluster.

06 In the Security section, check the Boot disk encryption attribute value. If the Boot disk encryption value is set to Google-managed, the nodes deployed for the selected GKE Autopilot cluster are encrypted with a Google-Managed Encryption Key instead of a Customer-Managed Encryption Key (CMEK).

07 Repeat steps no. 5 – 7 for each GKE Autopilot cluster created for the selected GCP project.

08 Repeat steps no. 2 – 8 for each GCP project deployed within your Google Cloud account.

Using GCP CLI

01 Run projects list command (Windows/macOS/Linux) with custom output filters to list the ID of each GCP project available in your Google Cloud account: 

gcloud projects list
	--format="table(projectId)"

02 The command output should return the requested GCP project IDS: 

PROJECT_ID
cc-web-project-123123
cc-dev-project-112233

03 Run container clusters list command (Windows/macOS/Linux) with the ID of the GCP project that you want to examine as the identifier parameter and custom output filters to describe the name and the region of each GKE Autopilot cluster provisioned for the selected project: 

gcloud container clusters list
	--project cc-web-project-123123
	--format="table(NAME,ZONE,AUTOPILOT)"

04 The command output should return the requested GKE Autopilot cluster identifiers. A GKE Autopilot cluster has AUTOPILOT set to {'enabled': True}: 

NAME: cc-auto-backend-cluster
ZONE: us-central1
AUTOPILOT: {'enabled': True}

NAME: cc-auto-frontend-cluster
ZONE: us-central1
AUTOPILOT: {'enabled': True}

05 Run container clusters describe command (Windows/macOS/Linux) with the name of the GKE Autopilot cluster that you want to examine as the identifier parameter, to describe the ID of the Customer-Managed Encryption Key (CMEK) used to encrypt the GKE node data: 

gcloud container clusters describe cc-auto-backend-cluster
	--region=us-central1
	--format="yaml(nodeConfig.bootDiskKmsKey)"

06 The command output should return the full resource ID of the CMEK used to encrypt the GKE node data: 

null

If the container clusters describe command output returns null, as shown in the example above, the nodes provisioned for the selected GKE Autopilot cluster are encrypted with a Google-Managed Encryption Key instead of a Customer-Managed Encryption Key (CMEK).

07 Repeat steps no. 5 and 6 for each GKE Autopilot cluster created for the selected GCP project.

08 Repeat steps no. 3 – 7 for each GCP project deployed in your Google Cloud account.

Remediation / Resolution

Case A: To enable encryption with Customer-Managed Encryption Keys (CMEKs) for your GKE Standard cluster nodes, you have to re-create the existing cluster node pools with the appropriate encryption configuration, by performing the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to access from the console top navigation bar.

03 To create and configure your new Customer-Managed Encryption Key (CMEK), perform the following actions:

  1. Navigate to Key management console available at https://console.cloud.google.com/security/kms.
  2. Before you can set up and configure your Customer-Managed Encryption Key (CMEK), you must create a key ring. A Cloud KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific location. To get started, choose CREATE KEY RING to set up the required key ring.
  3. A key ring requires a name and a location. On the Create key ring setup page, provide a unique name in the Key ring name box, select the key location type from the Location type list, then choose the appropriate key location from the Region/Multi-region dropdown list. The location can be either multi-region or associated with a particular region. If the CMEKs created later within this key ring will be used to encrypt/decrypt data in a particular region, select that region as the key ring location. Choose CREATE to deploy the new key ring.
  4. On the Create key setup page, provide the following information:
    1. For Name and protection level, provide a unique name for your new KMS key in the Key name box and choose the protection level that you want to use from the Protection Level dropdown list. Choose CONTINUE to continue the setup process.
    2. For Key material, choose Generated key to generate the key material for you (recommended). Choose CONTINUE.
    3. For Purpose and algorithm, choose Symmetric encrypt/decrypt to define the types of operations that your cryptographic key can perform. Choose CONTINUE to continue the setup.
    4. For Versions, configure the key rotation period as necessary. Choose CONTINUE.
    5. For Additional settings (optional), set the duration for the scheduled for destruction (i.e., soft deleted) state before the key is removed from the system. Choose ADD LABEL and use the Key and Value text fields to create labels in order to organize the identity of the new key.
    6. Choose CREATE to deploy your new Cloud KMS Customer-Managed Encryption Key (CMEK).

04 Navigate to Kubernetes Engine console available at https://console.cloud.google.com/kubernetes.

05 In the left navigation panel, under Resource Management, choose Clusters and select the OVERVIEW tab to access the list of GKE Standard clusters deployed for the selected GCP project.

06 Click on the name (link) of the GKE Standard cluster that you want to configure. A GKE Standard cluster has Mode set to Standard.

07 Select the NODES tab to access the node pools created for the selected cluster.

08 Click on the name (link) of the cluster node pool that you want to re-create and collect all the configuration information available for the selected resource.

09 Go back to the NODES panel and choose ADD NODE POOL to create a new node pool for your GKE cluster:

  1. For Node pool details, provide the following information:
    1. Provide a unique name for the new node pool in the Name box.
    2. Enter the number of nodes for the new pool in the Number of nodes (per zone) box.
    3. Choose whether or not to enable cluster auto-scaler. Must match the node pool configuration collected at step no. 8.
    4. (Optional) Choose whether or not to enable private nodes. Must match the node pool configuration collected at step no. 8.
    5. (Optional) If required, select the Specify node locations checkbox and choose additional zone nodes.
    6. (Optional) If required, configure the node pool upgrade strategy for the new node pool. Must match the node pool configuration collected at step no. 8.
  2. For Nodes, perform the following operations:
    1. Select the type of the node image from the Image type dropdown list.
    2. Choose the machine family, type, and series for the new node pool. Select the appropriate boot disk type and size. Must match the node pool configuration collected at step no. 10.
    3. For Boot disk encryption, select Cloud KMS key, and choose the Customer-Managed Encryption Key (CMEK) created at step no. 3 from the Select a Cloud KMS key dropdown list. Choose GRANT to grant the specified service account(s) the cloudkms.cryptoKeyEncrypterDecrypter role on the selected CMEK.
  3. For Networking, perform the following actions:
    1. Enter the maximum number of Kubernetes Pods per node in the Maximum Pods per node box.
    2. (Optional) Use Network tags to configure any required network tags.
    3. (Optional) Choose whether or not to create secondary address ranges. Must match the node pool configuration collected at step no. 8.
  4. For Security, provide the following information:
    1. Choose the appropriate service account from the Service account dropdown list.
    2. Select the appropriate access scope(s). Must match the node pool configuration collected at step no. 8.
    3. (Optional) Under Shielded options, enable integrity monitoring, secure boot, and confidential GKE nodes to enhance security at the cluster node level.
  5. For Metadata, add any required resource labels (tags), and configure the metadata settings such as GCE instance metadata based on the configuration information taken from the source node pool at step no. 8.
  6. Choose CREATE to create your new, CMEK-encrypted GKE node pool.

10 (Optional) Once the new GKE node pool is operating successfully, you can remove the source node pool in order to stop adding charges to your Google Cloud bill. Go back to the NODES panel and perform the following actions:

  1. Click on the name (link) of the node pool (i.e., source node pool) that you want to delete.
  2. Choose DELETE from to initiate the deletion process.
  3. In the confirmation box, confirm deletion by typing the node pool name, and choose DELETE to remove the node pool from your GKE cluster.

11 Repeat steps no. 8 - 10 to configure encryption for other node pools provisioned within the selected GKE cluster.

12 Repeat steps no. 6 – 11 for each GKE Standard cluster that you want to configure, created for the selected GCP project.

13 Repeat steps no. 2 – 12 for each GCP project available in your Google Cloud account.

Using GCP CLI

01 Before you can create your own Customer-Managed Encryption Key (CMEK), you have to provision a key ring. A Cloud KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific Google Cloud location. Run kms keyrings create command (Windows/macOS/Linux) to create a new Cloud KMS key ring in the specified location. If the keys deployed later within this key ring will be used to encrypt resources in a given region, select that region as the key ring location: 

gcloud kms keyrings create cc-cloud-gke-key-ring
	--location=us
	--project=cc-web-project-123123
	--format="table(name)"

02 The command output should return the resource name of the newly created key ring: 

NAME
projects/cc-web-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring

03 Run kms keys create command (Windows/macOS/Linux) to create a new Customer-Managed Encryption Key (CMEK) within the Cloud KMS key ring created at the previous steps: 

gcloud kms keys create cc-cloud-gke-kms-key
	--location=us
	--keyring=cc-cloud-gke-key-ring
	--purpose=encryption
	--protection-level=software
	--rotation-period=90d
	--next-rotation-time=2025-05-25T10:00:00.0000Z
	--format="table(name)"

04 The command output should return the full resource name of the new Customer-Managed Encryption Key: 

NAME
projects/cc-web-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-kms-key

05 Run kms keys add-iam-policy-binding command (Windows/macOS/Linux) to assign the CryptoKey Encrypter/Decrypter IAM role to the required service account: 

gcloud kms keys add-iam-policy-binding cc-cloud-gke-kms-key
	--keyring cc-cloud-gke-key-ring
	--location us-central1
	--member='serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com'
	--role roles/cloudkms.cryptoKeyEncrypterDecrypter

06 The command output should return the updated IAM policy (YAML format): 

Updated IAM policy for key [cc-cloud-gke-kms-key].
bindings:
- members:
	- serviceAccount:serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com
	role: roles/cloudkms.cryptoKeyEncrypterDecrypter
etag: ABCD1234ABCD1234
version: 1

07 Run container node-pools describe command (Windows/macOS/Linux) with the name of the GKE cluster node pool that you want to re-create as the identifier parameter and custom output filtering to describe the configuration information available for the selected node pool: 

gcloud container node-pools describe cc-gke-backend-pool-001
	--cluster=cc-gke-backend-cluster
	--region=us-central1
	--format=json

08 The command output should return the requested configuration information: 

{
	"config": {
		"diskSizeGb": 150,
		"diskType": "pd-standard",
		"imageType": "COS",
		"metadata": {
			"disable-legacy-endpoints": "true"
		},
		"serviceAccount": "default",
		"shieldedInstanceConfig": {
			"enableIntegrityMonitoring": true
		}
	},
	"locations": [
		"us-central1-a",
		"us-central1-b"
	],

	...

	"management": {
		"autoRepair": true,
		"autoUpgrade": true
	},
	"maxPodsConstraint": {
		"maxPodsPerNode": "110"
	},
	"name": "cc-gke-frontend-pool-001",
	"podIpv4CidrSize": 24,
	"status": "RUNNING",
	"upgradeSettings": {
		"maxSurge": 1
	}
}

09 Run container node-pools create command (Windows/macOS/Linux) with the information returned at the previous step as the configuration data, to create a new, CMEK-encrypted cluster node pool. Use the --boot-disk-kms-key paramater to specify your new Customer-Managed Encryption Key (CMEK), created at step no. 3: 

gcloud beta container node-pools create cc-new-backend-pool-001
	--cluster=cc-gke-backend-cluster
	--region=us-central1
	--disk-type=pd-standard
	--disk-size=150
	--boot-disk-kms-key=projects/cc-web-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-kms-key

10 The command output should return the full URL of the new cluster node pool: 

Created [https://container.googleapis.com/v1/projects/cc-web-project-12312/zones/us-central1/clusters/cc-gke-backend-cluster/nodePools/cc-new-backend-pool-001]

11 (Optional) Once the new node pool is operating successfully, you can remove the source node pool in order to stop adding charges to your GCP bill. Run container node-pools delete command (Windows/macOS/Linux) with the name of the resource that you want to remove as the identifier parameter, to remove the specified node pool from your GKE cluster: 

gcloud beta container node-pools delete cc-gke-backend-pool-001
	--cluster=cc-gke-backend-cluster
	--region=us-central1

12 Type Y to confirm the cluster node pool removal: 

The following node pool will be deleted.
[cc-gke-backend-pool-001] in cluster [cc-gke-backend-cluster] in [us-central1]
Do you want to continue (Y/n)?  Y

13 The output should return the container node-pools delete command request status: 

Deleting node pool cc-gke-backend-pool-001... done.
Deleted [https://container.googleapis.com/v1/projects/cc-web-project-12312/zones/us-central1/clusters/cc-gke-backend-cluster/nodePools/cc-gke-backend-pool-001].

14 Repeat steps no. 7 - 13 to configure encryption for other node pools provisioned for the selected GKE cluster.

15 Repeat steps no. 7 - 14 for each GKE cluster that you want to configure, available within the selected GCP project.

16 Repeat steps no. 1 – 15 for each GCP project deployed in your Google Cloud account.

Remediation / Resolution

Case B: To enable encryption with Customer-Managed Encryption Keys (CMEKs) for your GKE Autopilot cluster nodes, you have to re-create the existing cluster with the appropriate encryption configuration, by performing the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to access from the console top navigation bar.

03 To create and configure your new Customer-Managed Encryption Key (CMEK), perform the following actions:

  1. Navigate to Key management console available at https://console.cloud.google.com/security/kms.
  2. Before you can set up and configure your Customer-Managed Encryption Key (CMEK), you must create a key ring. A Cloud KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific location. To get started, choose CREATE KEY RING to set up the required key ring.
  3. A key ring requires a name and a location. On the Create key ring setup page, provide a unique name in the Key ring name box, select the key location type from the Location type list, then choose the appropriate key location from the Region/Multi-region dropdown list. The location can be either multi-region or associated with a particular region. If the CMEKs created later within this key ring will be used to encrypt/decrypt data in a particular region, select that region as the key ring location. Choose CREATE to deploy the new key ring.
  4. On the Create key setup page, provide the following information:
    1. For Name and protection level, provide a unique name for your new KMS key in the Key name box and choose the protection level that you want to use from the Protection Level dropdown list. Choose CONTINUE to continue the setup process.
    2. For Key material, choose Generated key to generate the key material for you (recommended). Choose CONTINUE.
    3. For Purpose and algorithm, choose Symmetric encrypt/decrypt to define the types of operations that your cryptographic key can perform. Choose CONTINUE to continue the setup.
    4. For Versions, configure the key rotation period as necessary. Choose CONTINUE.
    5. For Additional settings (optional), set the duration for the scheduled for destruction (i.e., soft deleted) state before the key is removed from the system. Choose ADD LABEL and use the Key and Value text fields to create labels in order to organize the identity of the new key.
    6. Choose CREATE to deploy your new Cloud KMS Customer-Managed Encryption Key (CMEK).

04 Navigate to Kubernetes Engine console available at https://console.cloud.google.com/kubernetes.

05 In the left navigation panel, under Resource Management, choose Clusters and select the OVERVIEW tab to access the list of GKE Standard clusters deployed for the selected GCP project.

06 Click on the name (link) of the GKE Autopilot cluster that you want to re-create. A GKE Autopilot cluster has Mode set to Autopilot.

07 Select DUPLICATE from the console top menu to initiate the setup process using the same configuration settings as the source cluster and perform the following changes:

  1. For Cluster basics, provide a unique name for the new GKE Autopilot cluster in the Name box.
  2. For Advanced settings, choose Security to show the security settings for the new GKE cluster. For Boot disk encryption, select Cloud KMS key, and choose the Customer-Managed Encryption Key (CMEK) created at step no. 3 from the Select a Cloud KMS key dropdown list. Choose GRANT to grant the specified service account(s) the cloudkms.cryptoKeyEncrypterDecrypter role on the selected CMEK.
  3. Choose CREATE from the console bottom menu to create your new, CMEK-encrypted cluster.

08 (Optional) Once the new GKE cluster is operating successfully, you can remove the source cluster in order to stop adding charges to your Google Cloud bill:

  1. Navigate back to the Clusters page and select the OVERVIEW tab to list the GKE clusters deployed for the selected GCP project.
  2. Select the GKE Autopilot cluster that you want to delete (i.e., source cluster), and choose DELETE from the console top menu.
  3. In the confirmation box, confirm deletion by typing the cluster name in the required text box, and choose DELETE to remove the selected GKE cluster.

09 Repeat steps no. 6 – 8 for each GKE Autopilot cluster that you want to re-create, created for the selected GCP project.

10 Repeat steps no. 2 – 8 for each GCP project available in your Google Cloud account.

Using GCP CLI

01 Run kms keyrings create command (Windows/macOS/Linux) to create a new Cloud KMS key ring in the specified location. If the keys deployed later within this key ring will be used to encrypt resources in a given region, select that region as the key ring location: 

gcloud kms keyrings create cc-cloud-gke-key-ring
	--location=us
	--project=cc-web-project-123123
	--format="table(name)"

02 The command output should return the resource name of the newly created key ring: 

NAME
projects/cc-web-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring

03 Run kms keys create command (Windows/macOS/Linux) to create a new Customer-Managed Encryption Key (CMEK) within the Cloud KMS key ring created at the previous steps: 

gcloud kms keys create cc-cloud-gke-kms-key
	--location=us
	--keyring=cc-cloud-gke-key-ring
	--purpose=encryption
	--protection-level=software
	--rotation-period=90d
	--next-rotation-time=2025-05-25T10:00:00.0000Z
	--format="table(name)"

04 The command output should return the full resource name of the new Customer-Managed Encryption Key: 

NAME
projects/cc-web-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-kms-key

05 Run kms keys add-iam-policy-binding command (Windows/macOS/Linux) to assign the CryptoKey Encrypter/Decrypter IAM role to the required service account: 

gcloud kms keys add-iam-policy-binding cc-cloud-gke-kms-key
	--keyring cc-cloud-gke-key-ring
	--location us-central1
	--member='serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com'
	--role roles/cloudkms.cryptoKeyEncrypterDecrypter

06 The command output should return the updated IAM policy (YAML format): 

Updated IAM policy for key [cc-cloud-gke-kms-key].
bindings:
- members:
	- serviceAccount:serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com
	role: roles/cloudkms.cryptoKeyEncrypterDecrypter
etag: ABCD1234ABCD1234
version: 1

07 Run container clusters describe command (Windows/macOS/Linux) with the name of the GKE Autopilot cluster that you want to re-create as the identifier parameter and custom output filtering to describe the configuration information available for the selected cluster: 

gcloud container clusters describe cc-auto-backend-cluster
	--region=us-central1
	--format="json"

08 The command output should return the requested configuration information (JSON format): 

{
	"addonsConfig": {
		"dnsCacheConfig": {
			"enabled": true
		},
		"gcePersistentDiskCsiDriverConfig": {
			"enabled": true
		},
		"gcpFilestoreCsiDriverConfig": {
			"enabled": true
		},
		"networkPolicyConfig": {
			"disabled": true
		}
	},
	"autopilot": {
		"enabled": true
	},

	...

	"releaseChannel": {
		"channel": "REGULAR"
	},
	"shieldedNodes": {
		"enabled": true
	},
	"status": "RUNNING",
	"subnetwork": "default",
	"verticalPodAutoscaling": {
		"enabled": true
	},
	"zone": "us-central1"
}

09 Run container clusters create-auto command (Windows/macOS/Linux) with the information returned at the previous step as the configuration data for the command parameters, to create a new, CMEK-encrypted GKE Autopilot cluster. Use the --boot-disk-kms-key paramater to specify your new Customer-Managed Encryption Key (CMEK), created at step no. 3: 

gcloud container clusters create-auto cc-new-auto-backend-cluster
	--project cc-web-project-123123
	--region us-central1
	--release-channel "regular"
	--network "projects/cc-web-project-123123/global/networks/default"
	--subnetwork "projects/cc-web-project-123123/regions/us-central1/subnetworks/default"
	--cluster-ipv4-cidr "/17"
	--services-ipv4-cidr "/22"
	--boot-disk-kms-key=projects/cc-web-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-kms-key

10 The command output should return the full URL of the new GKE Autopilot cluster: 

Created [https://container.googleapis.com/v1/projects/cc-web-project-123123/zones/us-central1/clusters/cc-new-auto-backend-cluster]

11 (Optional) Once the new GKE cluster is operating successfully, you can run container clusters delete command (Windows/macOS/Linux) to remove the source cluster in order to stop adding charges to your Google Cloud bill: 

gcloud container clusters delete cc-auto-backend-cluster
	--region=us-central1

12 Type Y (Yes) to confirm the removal: 

The following clusters will be deleted.
- [cc-auto-backend-cluster] in [us-central1]

Do you want to continue (Y/n)?  Y

13 The command output should return the URL of the deleted GKE cluster: 

Deleted [https://container.googleapis.com/v1/projects/cc-web-project-123123/zones/us-central1/clusters/cc-auto-backend-cluster]

14 Repeat step no. 7 – 13 for each GKE Autopilot cluster that you want to re-create, provisioned for the selected GCP project.

15 Repeat steps no. 1 – 14 for each GCP project deployed in your Google Cloud account.

References

Publication date May 4, 2021

Related GKE rules