Cluster Disks Encrypted with Customer-Managed Keys

01 Sign in to the Microsoft Azure Portal.

02 Navigate to All resources blade at https://portal.azure.com/#browse/all to access all your Microsoft Azure cloud resources.

03 Choose the Azure subscription that you want to access from the Subscription equalls all filter box and choose Apply.

04 From the Type equalls all filter box, choose Equals, select Kubernetes service, and choose Apply to list only the AKS clusters available in the selected Azure subscription.

05 Click on the name (link) of the AKS cluster that you want to examine.

06 In the resource navigation panel, under Settings, select Properties to access the configuration properties available for the selected AKS cluster.

07 In the Properties section, check for the Encryption type configuration properties to determine the type of the encryption key used for the selected AKS resource. If Encryption type is not set to Encryption at rest with customer-managed key, encryption at rest using Customer-Managed Keys (CMKs) is not enabled for your Azure Kubernetes Service (AKS) cluster disks.

08 Repeat steps no. 5 – 7 for each AKS cluster provisioned in the selected Azure subscription.

09 Repeat steps no. 3 – 8 for each subscription created in your Microsoft Azure cloud account.

01 Run account list command (Windows/macOS/Linux) with custom output filters to list the IDs of the cloud subscriptions available in your Azure cloud account:

az account list
  --query '[*].id'

02 The command output should return the requested subscription identifiers (IDs):

[
	"abcdabcd-1234-abcd-1234-abcdabcdabcd",
	"abcd1234-abcd-1234-abcd-abcd1234abcd"
]

03 Run account set command (Windows/macOS/Linux) with the ID of the Azure cloud subscription that you want to examine as the identifier parameter to set the selected subscription to be the current active subscription (the command does not produce an output):

az account set
  --subscription abcdabcd-1234-abcd-1234-abcdabcdabcd

04 Run aks list command (Windows/macOS/Linux) with custom query filters to list the name and the associated resource group for each Azure Kubernetes Service (AKS) cluster available in the selected Azure subscription:

az aks list
  --output table
  --query '[*].{name:name, resourceGroup:resourceGroup}'

05 The command output should return the requested AKS cluster names:

Name                    ResourceGroup
----------------------  ------------------------------
cc-data-mining-cluster  cloud-shell-storage-westeurope
cc-project5-cluster     cloud-shell-storage-westeurope

06 Run aks show command (Windows/macOS/Linux) with the name of the AKS cluster that you want to examine (and the associated resource group) as identifier parameters, to describe the ID of the disk encryption set configured for the selected AKS cluster. A disk encryption set allows you to manage encryption keys using Server-Side Encryption (SSE) for AKS cluster disks:

az aks show
  --name cc-data-mining-cluster
  --resource-group cloud-shell-storage-westeurope
  --query '{diskEncryptionSetId:diskEncryptionSetId}'

07 The command output should return the requested disk encryption set ID:

{
	"diskEncryptionSetId": null
}

08 Repeat steps no. 6 and 7 for each AKS cluster available within the selected Azure subscription.

09 Repeat steps no. 3 – 8 for each subscription created in your Microsoft Azure cloud account.

01 Sign in to the Azure Management Console.

02 Navigate to Key vaults blade available at https://portal.azure.com/#browse/Microsoft.KeyVault%2Fvaults.

03 Choose the Azure subscription that you want to access from the Subscription equalls all filter box and choose Apply.

04 Choose Create and perform the following actions to create the Azure key vault that will store your new Customer-Managed Key (CMK):

1. For Basics, choose the correct Azure subscription and resource group, provide a unique name for the new key vault, then select the Azure cloud region where the vault will be deployed and the appropriate pricing tier. Configure the vault retention period and enable purge protection. Both soft delete and purge protection must be enabled on the key vault. Choose Next to continue the setup process.
2. For Access configuration, select Vault access policy for Permission model, choose Create under Access policies, and follow the setup wizard to create the policy that allows Azure Kubernetes Service to create, get, recover, wrap, and unwrap encryption keys from the new vault. Once the access policy is configured, choose Create to create and attach it to the key vault. Configure the Azure resource access under Resource access. Choose Next to continue the setup.
3. For Networking, configure the network access control for the new key vault. You can connect to your new key vault either publicly, via public IP addresses or service endpoints, or privately, using a private endpoint. Choose Next to continue.
4. For Tags, use the Name and Value fields to create tags that will help organize the identity of the key vault. Choose Review + create to validate the key vault setup.
5. For Review + create, review the resource configuration details, then choose Create to create your new Azure key vault.

05 Once the deployment is complete, choose Go to resource to access your new Microsoft Azure key vault.

06 In the resource navigation panel, under Objets, select Keys, then choose Generate/Import to create the Customer-Managed Key required for AKS cluster disks encryption.

07 On the Create a key setup page, provide a unique name for the encryption key in the Name box, set Key type to RSA, RSA key size to 2048, choose an activation and/or expiration date, set the Enabled flag to Yes, then choose Create to generate your new Customer-Managed Key (CMK).

08 Once your new Customer-Managed Key is available, navigate to Disk Encryption Sets blade at https://portal.azure.com/#browse/Microsoft.Compute%2FdiskEncryptionSets.

09 Choose the Azure subscription that you want to access from the Subscription equalls all filter box and choose Apply.

10 Choose Create and perform the following actions to create a new disk encryption set. A disk encryption set allows you to manage encryption keys using Server-Side Encryption for AKS cluster disks:

1. For Basics, choose the appropriate subscription and resource group, provide a unique name for the new disk encryption set, select the Azure region where the encryption set will be deployed, choose Encryption at-rest with a customer-managed key for Encryption type, then choose the Azure key vault and Customer-Managed Key (CMK) created earlier in the Remediation process. Choose Next : Tags > to continue.
2. For Tags, use the Name and Value fields to create tags that will help organize the identity of the encryption set. Choose Next : Review + create > to validate the setup.
3. For Review + create, review the resource configuration details, then choose Create to create your new disk encryption set.

11 Navigate to Kubernetes Services blade available at https://portal.azure.com/#browse/Microsoft.ContainerService%2FmanagedClusters.

12 Select the Azure subscription that you want to access from the Subscription equalls all filter box and choose Apply.

13 Choose Create, select Kubernetes cluster, and perform the following actions to deploy your new AKS cluster:

1. For Basics, provide a name for the new cluster, select the resource group where the resource will be created, choose the appropriate cluster preset configuration, select the Azure region, Availability Zone(s), and Kubernetes version, and configure the authentication and authorization settings with Microsoft Entra ID and Azure RBAC or Kubernetes RBAC. Choose Next to continue the setup process.
2. For Node pools, configure the primary and optional node pools for the cluster and choose whether to enable virtual nodes. For Node pool OS disk encryption, set Encryption type to Encryption at-rest with a customer-managed key, and select the disk encryption set created earlier in the Remediation process from the Disk encryption set dropdown list. The disk encryption set select at this step will be used to encrypt the OS disks for all node pools in the cluster. Also, data disks will automatically be encrypted with the same disk encryption set, ensuring enhanced security.
3. For Networking, perform the following actions:
   1. To make the new AKS cluster private, check the Enable private cluster checkbox available under Private access.
   2. For Container networking, choose the Azure Container Networking Interface (CNI) mode that you want to use, check the Bring your own Azure virtual network checkbox and provide your Virtual Network (VNet) details. For Network policy, choose whether to use network policies to allow you to define rules for ingress and egress traffic between the pods within your cluster.
   3. Choose Next to continue the setup process.
4. For Integrations, connect your new AKS cluster with additional Azure cloud services such as Microsoft Defender for Cloud, Azure Container Registry, Istio, and Azure Policy. Choose Next to continue.
5. For Monitoring, you can enable and configure Container Insights, Prometheus, and Grafana for more comprehensive data on the overall performance and health of your AKS cluster. You can also configure any required alert rules. Choose Next to continue.
6. For Advanced, choose whether to use Azure Key Vault as a secrets store with your Kubernetes cluster via a CSI volume, provide the name of the necessary infrastructure resource group, and choose Next to continue the setup.
7. For Tags, create the necessary tags and choose Review + create to validate the configuration information provided during setup.
8. For Review + create, review the cluster configuration details, then choose Create to launch your new, CMK-encrypted AKS cluster. The provisioning process should take a few minutes to complete.

14 Repeat step no. 13 for each AKS cluster that you want to encrypt with a Customer-Managed Key (CMK), available in the selected subscription.

15 Repeat steps no. 2 – 14 for each subscription created in your Microsoft Azure cloud account.

01 Run account list command (Windows/macOS/Linux) with custom output filters to list the IDs of the cloud subscriptions available in your Azure cloud account:

az account list
  --query '[*].id'

02 The command output should return the requested subscription identifiers (IDs):

[
	"abcdabcd-1234-abcd-1234-abcdabcdabcd",
	"abcd1234-abcd-1234-abcd-abcd1234abcd"
]

03 Run account set command (Windows/macOS/Linux) with the ID of the Azure cloud subscription that you want to examine as the identifier parameter to set the selected subscription to be the current active subscription (the command does not produce an output):

az account set
  --subscription abcdabcd-1234-abcd-1234-abcdabcdabcd

04 Run keyvault create command (Windows/macOS/Linux) to create the Microsoft Azure key vault where the required Customer-Managed Key (CMK) will be placed. Both soft delete and purge protection must be enabled on the new key vault:

az keyvault create
  --name tm-project5-key-vault
  --resource-group cloud-shell-storage-westeurope
  --location westeurope
  --enable-rbac-authorization false
  --enabled-for-deployment true
  --enabled-for-template-deployment true
  --enable-purge-protection true

05 The command output should return the configuration information available for the new Azure key vault:

{
	"id": "/subscriptions/abcdabcd-1234-abcd-1234-abcdabcdabcd/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/tm-project5-vault",
	"location": "westeurope",
	"name": "tm-project5-vault",
	"properties": {
		"accessPolicies": [
			{
				"applicationId": null,
				"objectId": "abcdabcd-abcd-abcd-abcd-abcdabcdabcd",
				"permissions": {
					"certificates": [
						"all"
					],
					"keys": [
						"all"
					],
					"secrets": [
						"all"
					],
					"storage": [
						"all"
					]
				},
				"tenantId": "abcdabcd-1234-abcd-1234-abcdabcdabcd"
			}
		],
		"createMode": null,
		"enablePurgeProtection": true,
		"enableRbacAuthorization": false,
		"enableSoftDelete": true,
		"enabledForDeployment": true,
		"enabledForDiskEncryption": null,
		"enabledForTemplateDeployment": true,
		"hsmPoolResourceId": null,
		"networkAcls": null,
		"privateEndpointConnections": null,
		"provisioningState": "Succeeded",
		"publicNetworkAccess": "Enabled",
		"sku": {
			"family": "A",
			"name": "standard"
		},
		"softDeleteRetentionInDays": 30,
		"tenantId": "abcdabcd-1234-abcd-1234-abcdabcdabcd",
		"vaultUri": "https://tm-project5-vault.vault.azure.net/"
	},
	"resourceGroup": "cloud-shell-storage-westeurope",
	"systemData": {
		"createdAt": "2024-10-10T17:04:00.983000+00:00",
		"createdBy": "",
		"createdByType": "User",
		"lastModifiedAt": "2024-10-10T17:04:00.983000+00:00",
		"lastModifiedBy": "",
		"lastModifiedByType": "User"
	},
	"tags": {},
	"type": "Microsoft.KeyVault/vaults"
}

06 Run keyvault set-policy command (Windows/macOS/Linux) with the object ID (i.e. "properties.accessPolicies.objectId") and the name of the newly created key vault as the identifier parameters to assign the right permissions to the selected vault:

az keyvault set-policy
  --name tm-project5-key-vault
  --object-id abcdabcd-abcd-abcd-abcd-abcdabcdabcd
  --key-permissions create get recover unwrapKey wrapKey
  --query 'properties.accessPolicies'

07 The command output should return the modified key vault configuration information:

[
	{
		"applicationId": null,
		"objectId": "abcdabcd-abcd-abcd-abcd-abcdabcdabcd",
		"permissions": {
			"certificates": [
				"all"
			],
			"keys": [
				"recover",
				"unwrapKey",
				"get",
				"create",
				"wrapKey"
			],
			"secrets": [
				"all"
			],
			"storage": [
				"all"
			]
		},
		"tenantId": "abcd1234-abcd-1234-abcd-1234abcd1234"
	}
]

08 Run keyvault key create command (Windows/macOS/Linux) to create the Customer-Managed Key (CMK) necessary to encrypt data for your Azure OpenAI service instance:

az keyvault key create
  --name tm-project5-aks-key
  --vault-name tm-project5-key-vault
  --kty RSA
  --size 2048
  --ops decrypt encrypt sign unwrapKey verify wrapKey
  --protection software
  --disabled false
  --query 'key.kid'

09 The command output should return the URL of the new Customer-Managed Key:

"https://tm-project5-key-vault.vault.azure.net/keys/tm-project5-aks-key/abcd1234abcd1234abcd1234abcd1234"

10 Run az disk-encryption-set create command (Windows/macOS/Linux) to create the Azure disk encryption set required to encrypt the data on your AKS cluster disks:

az disk-encryption-set create
  --name cc-project5-encryption-set
  --key-url https://tm-project5-key-vault.vault.azure.net/keys/tm-project5-aks-key/abcd1234abcd1234abcd1234abcd1234
  --source-vault tm-project5-key-vault
  --resource-group cloud-shell-storage-westeurope
  --location westeurope
  --encryption-type EncryptionAtRestWithCustomerKey

11 The command output should return the information available for the newly created disk encryption set (including the encryption set ID):

{
	"activeKey": {
		"keyUrl": "https://tm-project5-key-vault.vault.azure.net/keys/tm-project5-aks-key/abcd1234abcd1234abcd1234abcd1234",
		"sourceVault": {
			"id": "/subscriptions/abcdabcd-1234-abcd-1234-abcdabcdabcd/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/tm-project5-key-vault"
		}
	},
	"autoKeyRotationError": null,
	"encryptionType": "EncryptionAtRestWithCustomerKey",
	"federatedClientId": null,
	"id": "/subscriptions/abcdabcd-1234-abcd-1234-abcdabcdabcd/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Compute/diskEncryptionSets/cc-project5-encryption-set",
	"identity": {
		"principalId": "abcdabcd-1234-abcd-1234-abcd1234abcd",
		"tenantId": "abcdabcd-1234-abcd-1234-abcd1234abcd",
		"type": "SystemAssigned",
		"userAssignedIdentities": null
	},
	"lastKeyRotationTimestamp": null,
	"location": "westeurope",
	"name": "cc-project5-encryption-set",
	"previousKeys": null,
	"provisioningState": "Succeeded",
	"resourceGroup": "cloud-shell-storage-westeurope",
	"rotationToLatestKeyVersionEnabled": null,
	"tags": null,
	"type": "Microsoft.Compute/diskEncryptionSets"
}

12 Run aks create command (OSX/Linux/UNIX) to launch a new AKS cluster in the selected Azure subscription. Include the --node-osdisk-diskencryptionset-id command parameter to specify the disk encryption set created at the previous steps. This will enable encryption at rest for your AKS cluster disks using Customer-Managed Keys (CMKs):

az aks create
  --name cc-new-data-mining-cluster
  --resource-group cloud-shell-storage-westeurope
  --location "westeurope"
  --node-count 1
  --node-vm-size "Standard_B2s"
  --node-osdisk-size 50
  --kubernetes-version "1.29.8"
  --network-plugin azure
  --generate-ssh-keys
  --tier standard
  --node-osdisk-diskencryptionset-id "/subscriptions/abcdabcd-1234-abcd-1234-abcdabcdabcd/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Compute/diskEncryptionSets/cc-project5-encryption-set"

13 The command output should return the configuration information available for the new AKS cluster:

{
	"aadProfile": null,
	"addonProfiles": {
		"azureKeyvaultSecretsProvider": {
			"config": null,
			"enabled": false,
			"identity": null
		},
		"azurepolicy": {
			"config": null,
			"enabled": false,
			"identity": null
		}
	},
	"apiServerAccessProfile": null,
	"autoUpgradeProfile": {
		"upgradeChannel": "none"
	},

	...

	"storageProfile": {
		"blobCsiDriver": null,
		"diskCsiDriver": {
			"enabled": true
		},
		"fileCsiDriver": {
			"enabled": true
		},
		"snapshotController": {
			"enabled": true
		}
	},
	"supportPlan": "KubernetesOfficial",
	"systemData": null,
	"type": "Microsoft.ContainerService/ManagedClusters",
	"windowsProfile": null,
	"workloadAutoScalerProfile": {
		"keda": null
	}
}

14 Repeat steps no. 12 and 13 for each AKS cluster that you want to encrypt with a Customer-Managed Key (CMK), available in the selected subscription.

15 Repeat steps no. 3 – 14 for each subscription created in your Microsoft Azure cloud account.