Use Private Key Vaults for Encryption at Rest in Azure Kubernetes Service (AKS)

Risk Level: High (not acceptable risk)

Ensure that your Azure Kubernetes Service (AKS) clusters are configured with encryption at rest for Kubernetes secrets in etcd using a private Azure Key Vault.

Security

Operational
excellence

Cost
optimisation

Enabling encryption at rest for Kubernetes secrets in etcd using a private Key Vault adds a layer of security, protecting sensitive data, meeting compliance requirements, and preventing unauthorized access. It ensures that even if an attacker gains access to storage, encrypted secrets remain protected and inaccessible without the encryption keys. Enabling encryption with a private Key Vault in AKS automatically creates a private endpoint and private link in the node resource group. It establishes a private endpoint connection between the Key Vault and the AKS cluster.

Audit

To determine if your Azure Kubernetes clusters are using a private Key Vault for secret data encryption, perform the following operations:

Checking Azure Kubernetes Service (AKS) clusters for encryption with private Key Vaults using Azure Console (Azure Portal) is not currently supported.

Using Azure CLI

Run aks list command (Windows/macOS/Linux) using custom query filters to list the name and the associated resource group for each Azure Kubernetes Service (AKS) cluster available in the current subscription:
```
az aks list
  --output table
  --query '[*].{name:name, resourceGroup:resourceGroup}'
```

The command output should return the requested AKS cluster names:

Name                     ResourceGroup
----------------------   ------------------------------
cc-project5-aks-cluster  cloud-shell-storage-westeurope
cc-data-mining-cluster   cloud-shell-storage-westeurope

Run aks show command (Windows/macOS/Linux) using the name of the AKS cluster that you want to examine and its associated resource group as the identifier parameters to describe the Microsoft Entra ID integration profile available for the selected AKS cluster:
```
az aks show
  --name cc-project5-aks-cluster
  --resource-group cloud-shell-storage-westeurope
  --query 'securityProfile.azureKeyVaultKms.keyVaultNetworkAccess'
```
The command output should return the identity type used (system-assigned or user-assigned):
```
"Public"
```
If the aks show command output returns "Public", as shown in the output example above, the selected Azure Kubernetes Service (AKS) cluster is not using a private Key Vault for secret data encryption.
Repeat steps no. 3 and 4 for each AKS cluster available within the current Azure subscription.
Repeat steps no. 1 – 4 for each subscription created in your Microsoft Azure cloud account.

Remediation / Resolution

To configure Azure Kubernetes Service (AKS) clusters to use a private Key Vault for secret data encryption, perform the following operations:

Configuring Azure Kubernetes clusters to use a private Key Vault for secret data encryption using Azure Console (Azure Portal) is not currently supported.

Using Azure CLI

Run keyvault create command (OSX/Linux/UNIX) to create a new private Azure Key Vault:

az keyvault create 
  --name cc-private-key-vault 
  --resource-group cloud-shell-storage-westeurope 
  --location westeurope 
  --public-network-access Disabled 
  --query 'id'

The command output should return the ID of the new private Key Vault:

"/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/cc-private-key-vault"

Run identity create command (OSX/Linux/UNIX) to create the required user-assigned managed identity:

az identity create 
  --name cc-user-assigned-identity 
  --resource-group cloud-shell-storage-westeurope 
  --query 'principalId'

The command output should return the ID of the new user-assigned managed identity:
```
"1234abcd-abcd-1234-abcd-abcd1234abcd"
```

Run keyvault set-policy command (OSX/Linux/UNIX) to assign the appropriate permissions to the private Key Vault:

az keyvault set-policy 
  --name cc-private-key-vault 
  --key-permissions decrypt encrypt 
  --object-id "1234abcd-abcd-1234-abcd-abcd1234abcd" 
  --query 'properties.accessPolicies'

The command output should return the new Key Vault policy:

[
	{
		"applicationId": null,
		"objectId": "abcd1234-abcd-1234-abcd-1234abcd1234",
		"permissions": {
			"certificates": [
				"all"
			],
			"keys": [
				"all"
			],
			"secrets": [
				"all"
			],
			"storage": [
				"all"
			]
		},
		"tenantId": "abcd1234-abcd-1234-abcd-1234abcd1234"
	},
	{
		"applicationId": null,
		"objectId": "1234abcd-abcd-1234-abcd-abcd1234abcd",
		"permissions": {
			"certificates": null,
			"keys": [
				"decrypt",
				"encrypt"
			],
			"secrets": null,
			"storage": null
		},
		"tenantId": "abcd1234-abcd-1234-abcd-1234abcd1234"
	}
]

Run role assignment create command (OSX/Linux/UNIX) to assign the Key Vault Contributor role in order to create a private link between the private Key Vault and the AKS cluster:

az role assignment create 
  --role "Key Vault Contributor" 
  --assignee-object-id "1234abcd-abcd-1234-abcd-abcd1234abcd" 
  --assignee-principal-type "ServicePrincipal" 
  --scope "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/cc-private-key-vault"

The command output should return the new assignment information:

{
	"condition": null,
	"conditionVersion": null,
	"createdBy": null,
	"createdOn": "2023-07-17T17:33:28.931105+00:00",
	"delegatedManagedIdentityResourceId": null,
	"description": null,
	"id": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/cc-private-key-vault/providers/Microsoft.Authorization/roleAssignments/1234abcd-abcd-1234-abcd-abcd1234abcd",
	"name": "1234abcd-abcd-1234-abcd-abcd1234abcd",
	"principalId": "1234abcd-abcd-1234-abcd-abcd1234abcd",
	"principalType": "ServicePrincipal",
	"resourceGroup": "cloud-shell-storage-westeurope",
	"roleDefinitionId": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/providers/Microsoft.Authorization/roleDefinitions/1234abcd-abcd-1234-abcd-abcd1234abcd",
	"scope": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/cc-private-key-vault",
	"type": "Microsoft.Authorization/roleAssignments",
	"updatedBy": "1234abcd-abcd-1234-abcd-abcd1234abcd",
	"updatedOn": "2023-07-17T17:33:29.143905+00:00"
}

Run keyvault key create command (OSX/Linux/UNIX) to create the required Key Vault key:

az keyvault key create 
  --name cc-private-kms-key 
  --vault-name cc-private-key-vault 
  --query 'key.kid'

The command output should return the ID of the new Azure Key Vault key:

"https://cc-private-key-vault.vault.azure.net/keys/cc-private-kms-key/1234abcd-abcd-1234-abcd-abcd1234abcd"

Run aks update command (OSX/Linux/UNIX) using the name of the cluster that you want to configure as the identifier parameter, to enable encryption at rest for Kubernetes secrets in etcd using a private Key Vault:

az aks update 
  --name cc-project5-aks-cluster 
  --resource-group cloud-shell-storage-westeurope 
  --enable-azure-keyvault-kms 
  --azure-keyvault-kms-key-id "https://cc-private-key-vault.vault.azure.net/keys/cc-private-kms-key/1234abcd-abcd-1234-abcd-abcd1234abcd" 
  --azure-keyvault-kms-key-vault-network-access "Private"
  --azure-keyvault-kms-key-vault-resource-id "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.KeyVault/vaults/cc-private-key-vault"

Once the update process is completed, the command output should return the information available for the modified AKS cluster:

{
	"aadProfile": {
		"adminGroupObjectIds": [
			"abcd1234-abcd-1234-abcd-1234abcd1234"
		],
		"clientAppId": null,
		"managed": true,
		"serverAppId": null,
		"serverAppSecret": null,
		"tenantId": "1234abcd-abcd-1234-abcd-abcd1234abcd"
	},
	"apiServerAccessProfile": null,
	"autoUpgradeProfile": {
		"upgradeChannel": "none"
	},
	"identity": {
		"principalId": "abcd1234-abcd-1234-abcd-1234abcd1234",
		"tenantId": "1234abcd-abcd-1234-abcd-abcd1234abcd",
		"type": "SystemAssigned",
		"userAssignedIdentities": null
	},
	"azureMonitorProfile": null,
	"currentKubernetesVersion": "1.26.3",
	"disableLocalAccounts": false,
	"diskEncryptionSetId": null,
	"dnsPrefix": "cc-project5-aks-cluster-dns",
	"enablePodSecurityPolicy": null,
	"enableRbac": true,
	"extendedLocation": null,
	"fqdn": "cc-project5-aks-cluster-dns-abcd1234.hcp.westeurope.azmk8s.io",
	"fqdnSubdomain": null,
	"httpProxyConfig": null,
	"kubernetesVersion": "1.26.3",
	"linuxProfile": null,
	"location": "westeurope",
	"maxAgentPools": 100,
	"name": "cc-project5-aks-cluster",
	"nodeResourceGroup": "MC_cloud-shell-storage-westeurope_cc-project5-aks-cluster_westeurope",
	"oidcIssuerProfile": {
		"enabled": false,
		"issuerUrl": null
	},
	"podIdentityProfile": null,
	"powerState": {
		"code": "Running"
	},
	"privateFqdn": null,
	"privateLinkResources": null,
	"provisioningState": "Succeeded",
	"publicNetworkAccess": null,
	"resourceGroup": "cloud-shell-storage-westeurope",
	"servicePrincipalProfile": {
		"clientId": "msi",
		"secret": null
	},
	"sku": {
		"name": "Base",
		"tier": "Free"
	},
	"storageProfile": {
		"blobCsiDriver": null,
		"diskCsiDriver": {
			"enabled": true
		},
		"fileCsiDriver": {
			"enabled": true
		},
		"snapshotController": {
			"enabled": true
		}
	},
	"supportPlan": "KubernetesOfficial",
	"systemData": null,
	"tags": null,
	"type": "Microsoft.ContainerService/ManagedClusters",
	"windowsProfile": null,
	"workloadAutoScalerProfile": {
		"keda": null
	}
}

Run kubectl get secrets command (OSX/Linux/UNIX) to update all cluster secrets:
```
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
```
Repeat steps no. 11 - 13 for each Azure Kubernetes Service cluster that you want to configure, available within the current subscription.
Repeat steps no. 1 – 14 for each subscription created in your Microsoft Azure cloud account.

References

Azure Official Documentation
Add Key Management Service (KMS) etcd encryption to an Azure Kubernetes Service (AKS) cluster
Add secret management

Publication date Aug 8, 2023

Audit

Using Azure CLI

Remediation / Resolution

Using Azure CLI

References

Related AKS rules