Enable In-Transit Encryption

Risk Level: Medium (should be achieved)

Ensure that your Amazon MSK clusters are configured to use the Transport Layer Security (TLS) protocol to encrypt your data as it travels between the brokers within the MSK clusters and between Apache Kafka clients and clusters. In-transit encryption prevents potential attackers from intercepting cluster traffic and keeps your Amazon MSK data secure.

Security

When working with production and business-critical data, it is strongly recommended to enforce in-transit encryption to protect your data from unauthorized access and fulfill compliance requirements for data encryption within your organization. For example, a compliance requirement is to protect sensitive data that could potentially identify a specific individual such as Personally Identifiable Information (PII), usually used in Financial Services, Healthcare, and Telecommunications sectors.

Audit

To determine if in-transit encryption is enabled for your Amazon MSK clusters, perform the following operations:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon MSK console at https://console.aws.amazon.com/msk.

03 In the main navigation panel, underMSK Clusters, choose Clusters.

04 Click on the name (link) of the cluster that you want to examine, available in the Cluster namecolumn.

05 Select the Properties tab and check the Encrypt data in transit feature configuration available in the Encryption section:

For Within the cluster, if the TLS encryption attribute value is set to Not enabled, data-in-transit encryption is not enabled for the selected Amazon MSK cluster, therefore the communication between MSK cluster brokers is not protected.
For Between clients and brokers, if the TLS encryption attribute is set to Not enabled, data-in-transit encryption is not enabled for the communication between Amazon MSK cluster clients and brokers.

06 Repeat step no. 4 and 5 for each Amazon Managed Streaming for Kafka (MSK) cluster provisioned within the current AWS region.

07 Change the AWS cloud region from the navigation bar and repeat the Audit process for the other regions.

Using AWS CLI

01 Runlist-clusters command (OSX/Linux/UNIX) using custom query filters to list the Amazon Resource Names (ARNs) of the Amazon MSK clusters available in the selected AWS region:

aws kafka list-clusters
  --region us-east-1
  --query 'ClusterInfoList[*].ClusterArn'

02 The command output should return an array with the requested cluster ARNs:

[
	"arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab",
	"arn:aws:kafka:us-east-1:123456789012:cluster/cc-msk-app-cluster/aabbccdd-1234-aabb-1234-aabbccddaabb-cd"
]

03 Rundescribe-cluster command (OSX/Linux/UNIX) using the ARN of the Amazon MSK cluster that you want to examine as the identifier parameter and custom query filters to describe the in-transit encryption configuration information available for the selected cluster:

aws kafka describe-cluster
  --region us-east-1
  --cluster-arn arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab
  --query 'ClusterInfo.EncryptionInfo.EncryptionInTransit'

04 The command output should return**the requested configuration information:

{
	"ClientBroker": "PLAINTEXT",
	"InCluster": false
}

If the "ClientBroker" attribute value is set to "PLAINTEXT", as shown in the output example above, in-transit encryption is not enabled for the communication between Amazon MSK cluster clients and brokers. If the value returned for the "InCluster" attribute value is false, in-transit encryption is not enabled within the selected Amazon MSK cluster, therefore the communication between cluster brokers is not protected against eavesdropping.

05 Repeat steps no. 3 and 4 for each Amazon Managed Streaming for Kafka (MSK) cluster available in the selected AWS region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the Audit process for other regions.

Remediation / Resolution

Encryption in transit can't be enabled for existing Amazon MSK clusters. To encrypt the data as it travels between the broker nodes and between Apache Kafka clients and cluster, you must re-create your MSK cluster with the appropriate encryption configuration. To relaunch your Amazon MSK cluster with protection against eavesdropping, perform the following operations:

Using AWS CloudFormation

01 CloudFormation template (JSON):

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Enable In-Transit Encryption",
	"Resources": {
		"MSKCluster": {
			"Type": "AWS::MSK::Cluster",
			"Properties": {
				"ClusterName": "cc-encrypted-msk-cluster",
				"KafkaVersion": "3.2.0",
				"NumberOfBrokerNodes": 2,
				"EnhancedMonitoring": "DEFAULT",
				"BrokerNodeGroupInfo": {
					"BrokerAZDistribution": "DEFAULT",
					"ClientSubnets": [
						"subnet-0abcd1234abcd1234",
						"subnet-01234abcd1234abcd"
					],
					"InstanceType": "kafka.m5.large",
					"SecurityGroups": [
						"sg-0abcd1234abcd1234"
					],
					"StorageInfo": {
						"EbsStorageInfo": {
							"VolumeSize": 500
						}
					}
				},
				"EncryptionInfo": {
					"EncryptionInTransit": {
						"ClientBroker": "TLS",
						"InCluster": true
					}
				}
			}
		}
	}
}

02 CloudFormation template (YAML):

AWSTemplateFormatVersion: '2010-09-09'
	Description: Enable In-Transit Encryption
	Resources:
	MSKCluster:
		Type: AWS::MSK::Cluster
		Properties:
		ClusterName: cc-encrypted-msk-cluster
		KafkaVersion: 3.2.0
		NumberOfBrokerNodes: 2
		EnhancedMonitoring: DEFAULT
		BrokerNodeGroupInfo:
			BrokerAZDistribution: DEFAULT
			ClientSubnets:
			- subnet-0abcd1234abcd1234
			- subnet-01234abcd1234abcd
			InstanceType: kafka.m5.large
			SecurityGroups:
			- sg-0abcd1234abcd1234
			StorageInfo:
			EbsStorageInfo:
				VolumeSize: 500
		EncryptionInfo:
			EncryptionInTransit:
			ClientBroker: TLS
			InCluster: true

Using Terraform (AWS Provider)

01 Terraform configuration file (.tf):

terraform {
	required_providers {
		aws = {
			source  = "hashicorp/aws"
			version = "~> 4.0"
		}
	}

	required_version = ">= 0.14.9"
}

provider "aws" {
	profile = "default"
	region  = "us-east-1"
}

resource "aws_msk_cluster" "msk-cluster" {
	cluster_name           = "cc-encrypted-msk-cluster"
	kafka_version          = "3.2.0"
	number_of_broker_nodes = 2
	enhanced_monitoring    = "DEFAULT"

	broker_node_group_info {
		instance_type  = "kafka.m5.large"
		client_subnets = ["subnet-0abcd1234abcd1234","subnet-01234abcd1234abcd"]
		storage_info {
			ebs_storage_info {
			volume_size = 500
			}
		}
		security_groups = ["sg-0abcd1234abcd1234"]
	}

	# Enable In-Transit Encryption
	encryption_info {
		encryption_in_transit {
			client_broker = "TLS"
			in_cluster    = true
		}
	}

}

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon MSK console at https://console.aws.amazon.com/msk.

03 In the main navigation panel, underMSK Clusters, choose Clusters.

04 Click on the name (link) of the cluster that you want to re-create, available in the Cluster name column.

05 Select the Properties tab and collect all the configuration information available for the selected cluster.

06 Navigate back to the Clusters page and choose Create cluster to launch a new Amazon MSK cluster.

07 On the Create cluster setup page, perform the following actions:

For Step 1 Cluster settings, perform the following:
- For Cluster creation method, choose Custom create.
- Provide a unique name for the new cluster in the Cluster name box.
- For Cluster type, choose the type of cluster that you want to create. Must match the type of the source (original) cluster. If your source cluster type is Provisioned, configure the brokers as described below.
- Choose the Apache Kafka version that you want to use for the cluster brokers from the Apache Kafka version dropdown list.
- In the Brokers section, configure the broker nodes for your new MSK cluster. Choose the right broker type, the number of zones, and the number of brokers per zone. The broker node configuration must match the broker configuration information collected at step no. 5.
- In the Storage section, you can configure the storage volume and storage throughput per broker node.
- In the Configuration section, choose whether to use the default configuration provided by Amazon MSK or to use your own, custom configuration.
- Choose Next to continue the setup process.
For Step 2 Networking, choose VPC network, the Availability Zones (AZs), and the subnets where you want Amazon MSK to deploy the brokers for your new cluster. For Security groups in Amazon EC2, select one or more security groups to assign to the cluster's ENIs. The networking configuration must match the one identified at step no. 5. Choose Next to continue the setup.
For Step 3 Security, perform the following actions:
- For Access control methods, choose the method that you want Amazon MSK to use to authenticate clients and allow or deny actions.
- For Encrypt data in transit, select TLS encryption under Between clients and brokers to encrypt the communication between cluster clients and brokers, and choose TLS encryption under Within the cluster to encrypt the communication between the cluster brokers.
- For Encrypt data at rest, choose Use customer managed key and follow the steps outlined in this conformity rule to enable encryption at rest using a KMS customer-managed Customer Master Key (CMK).
- Choose Next to continue the setup.
For Step 4 Monitoring and tags, configure the monitoring options for the new cluster. The monitoring configuration must match the monitoring information collected at step no. 5.
(Optional) To attach tags to your new cluster, use the Add new tag button available in the Cluster tags – optional section. Choose Next to continue.
For Step 5 Review and create, review the cluster configuration, then choose Create cluster to launch your new Amazon MSK cluster.

08 Repeat steps no. 4 – 7 for each Amazon MSK cluster that you want to re-create, available within the current AWS region.

09 Change the AWS cloud region from the navigation bar and repeat the Remediation process for the other regions.

Using AWS CLI

01 Rundescribe-cluster command (OSX/Linux/UNIX) using the ARN of the Amazon MSK cluster that you want to re-create as the identifier parameter and custom query filters to get the configuration information available for the selected cluster:

aws kafka describe-cluster
  --region us-east-1
  --cluster-arn arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab

02 The command output should return**the requested configuration information:

{
	"ClusterInfo": {
		"BrokerNodeGroupInfo": {
			"BrokerAZDistribution": "DEFAULT",
			"ClientSubnets": [
				"subnet-0abcd1234abcd1234",
				"subnet-01234abcd1234abcd"
			],
			"InstanceType": "kafka.m5.large",
			"SecurityGroups": [
				"sg-0abcd1234abcd1234"
			],
			"StorageInfo": {
				"EbsStorageInfo": {
					"VolumeSize": 500
				}
			}
		},

		...


		"EncryptionInfo": {
			"EncryptionAtRest": {
				"DataVolumeKMSKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcdabcd-1234-1234-1234-abcdabcdabcd"
			},
			"EncryptionInTransit": {
				"ClientBroker": "PLAINTEXT",
				"InCluster": false
			}
		},
		"OpenMonitoring": {
			"Prometheus": {
				"JmxExporter": {
					"EnabledInBroker": false
				},
				"NodeExporter": {
					"EnabledInBroker": false
				}
			}
		}
	}
}

03 Define the cluster broker node configuration using the information returned at the previous step and save the configuration document to a JSON file named cc-broker-node-group-config.json:

{
	"BrokerAZDistribution": "DEFAULT",
	"ClientSubnets": [
			"subnet-0abcd1234abcd1234",
			"subnet-01234abcd1234abcd"
	],
	"InstanceType": "kafka.m5.large",
	"SecurityGroups": [
		"sg-0abcd1234abcd1234"
	],
	"StorageInfo": {
		"EbsStorageInfo": {
			"VolumeSize": 500
		}
	}
}

04 Define the cluster encryption configuration and save the configuration document to a JSON file named cc-encryption-config.json. The following example enables in-transit encryption for the communication between the cluster brokers, between cluster clients and brokers, and enables encryption at rest using a customer-managed Customer Master Key (CMK):

{
	"EncryptionInTransit": {
	"ClientBroker": "TLS",
		"InCluster": true
	},
	"EncryptionAtRest": {
		"DataVolumeKMSKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcdabcd-1234-1234-1234-abcdabcdabcd"
	}
}

05 Runcreate-cluster command (OSX/Linux/UNIX) to launch a new Amazon MSK cluster using the configuration parameters defined at the previous steps (i.e. cc-broker-node-group-config.json and cc-encryption-config.json):

aws kafka create-cluster
  --cluster-name cc-encrypted-msk-cluster
  --kafka-version "2.6.2"
  --number-of-broker-nodes 2
  --enhanced-monitoring DEFAULT
  --broker-node-group-info file://cc-broker-node-group-config.json
  --encryption-info file://cc-encryption-config.json

06 The command output should return the information available for the new Amazon MSK cluster:

{
	"ClusterArn": "arn:aws:kafka:us-east-1:123456789012:cluster/cc-encrypted-msk-cluster/1234abcd-1234-abcd-1234-abcd1234abcd-ad",
	"ClusterName": "cc-encrypted-msk-cluster",
	"State": "CREATING"
}

07 Repeat steps no. 1 – 6 for each Amazon MSK cluster that you want to re-create, available in the selected AWS region.

08 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 7 to perform the Remediation process for other regions.

References

AWS Command Line Interface (CLI) Documentation
kafka
list-clusters
describe-cluster
create-cluster

Publication date Jul 6, 2022

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS CloudFormation

Using Terraform (AWS Provider)

Using AWS Console

Using AWS CLI

References

Related MSK rules