Enable Pub/Sub Topic Encryption with Customer-Managed Encryption Key

01 Sign in to Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to access from the console top navigation bar.

03 Navigate to Pub/Sub console available at https://console.cloud.google.com/cloudpubsub.

04 In the navigation panel, select Topics to access the Pub/Sub topics created for the selected project.

05 Click on the ID (link) of the Pub/Sub topic that you want to examine.

06 Select the DETAILS tab and check the Encryption key configuration attribute value. If the Encryption key attribute value is set to Google-managed, the messages published to the selected Google Cloud Pub/Sub topic are not encrypted using a Customer-Managed Encryption Key (CMEK).

07 Repeat steps no. 5 and 6 for each Pub/Sub topic created within the selected GCP project.

08 Repeat steps no. 2 – 7 for each project deployed within your Google Cloud account.

01 Run projects list command (Windows/macOS/Linux) with custom query filters to list the IDs of all the Google Cloud Platform (GCP) projects available in your cloud account:

gcloud projects list --format="table(projectId)"

02 The command output should return the requested GCP project identifiers:

PROJECT_ID
cc-dataflow-project-123123
cc-bigdata-project-112233

03 Run pubsub topics list command (Windows/macOS/Linux) with custom filtering to describe the fully qualified identifier of each Pub/Sub topic deployed within the selected Google Cloud project:

gcloud pubsub topics list
  --project cc-dataflow-project-123123
  --format="table(name)"

04 The command output should return the requested topic identifier(s):

NAME
projects/cc-dataflow-project-123123/topics/cc-dataflow-app-topic
projects/cc-dataflow-project-123123/topics/cc-dataflow-web-topic

05 Run pubsub topics describe command (Windows/macOS/Linux) using the fully qualified identifier of the Google Cloud Pub/Sub topic that you want to examine as the identifier parameter and custom query filters to describe the resource ID of the Customer-Managed Encryption Key (CMEK) used to encrypt the topic messages:

gcloud pubsub topics describe projects/logical-lock-345715/topics/cc-dataflow-app-topic
  --format="json(kmsKeyName)"

06 The command output should return the resource ID of the requested Customer-Managed Encryption Key:

null

07 Repeat steps no. 5 and 6 for each Pub/Sub topic available within the selected project.

08 Repeat steps no. 3 – 7 for each GCP project deployed in your Google Cloud account.

01 Sign in to Google Cloud Management Console.

02 Select the GCP project that you want to access from the console top navigation bar.

03 To create and configure your new Customer-Managed Encryption Key (CMEK), perform the following actions:

1. Navigate to Key Management Service (KMS) console available at https://console.cloud.google.com/security/kms.
2. Before you can set up and manage any Customer-Managed Encryption Keys (CMEKs), you must create a key ring. A KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific location. Choose CREATE KEY RING from the top menu to set up the required key ring and the new Customer-Managed Encryption Key (CMEK).
3. A key ring requires a name and location. On the Create key ring page, provide a unique name in the Key ring name box, select the appropriate Location type, then choose a location for the key ring from the Region dropdown list. If the CMEKs created later within the key ring will be used to encrypt/decrypt resources in a given region, select that region as the key ring location. Choose CREATE to deploy the new key ring.
4. On the Create key setup page, provide a name for your new key in the Key name box, choose the protection level that you want to use, choose Generated key for Key material, select Symmetric encrypt/decrypt from the Purpose dropdown list to define the types of operations that your cryptographic key can perform, and configure the key rotation parameters. Choose CREATE to deploy your new Cloud KMS Customer-Managed Encryption Key (CMEK).

04 Navigate to Pub/Sub console available at https://console.cloud.google.com/cloudpubsub.

05 In the navigation panel, select Topics to access the Pub/Sub topics created for the selected project.

06 Click on the ID (link) of the Pub/Sub topic that you want to re-create and collect all the configuration information available for the selected resource such as subscription settings and labels.

07 Select the DETAILS tab and check the Encryption key configuration attribute value.

08 Go back to the Topics page and choose CREATE TOPIC from the top menu to initiate the topic setup process.

09 On the Create topic setup page, perform the following actions:

1. Provide a unique identifier for the new topic in the Topic ID box.
2. In the Encryption section, choose Customer-managed encryption key (CMEK), and select the CMEK created at step no. 3 from the Select a customer-managed key dropdown list. If the newly created CMEK does not appear in the dropdown list, select ENTER KEY MANUALLY and provide the full resource ID of your Customer-Managed Encryption Key (CMEK).
3. Inside The service account does not have the "cloudkms.cryptoKeyEncrypterDecrypter" role. Verify the service account has permission to encrypt/decrypt with the selected key. box, choose Grant to grant the specified service account the required IAM role on the selected CMEK.
4. Choose CREATE to deploy your new Google Cloud Pub/Sub topic.

10 Click on the newly created Pub/Sub topic, select the SUBSCRIPTIONS tab, and choose CREATE SUBSCRIPTIONto create the required subscription. Use the information collected at step no. 6 to configure the new topic subscription. Repeat this step to create as many subscriptions as needed.

11 If required, update your application to point to the new Pub/Sub topic and subscriptions.

12 To stop adding unnecessary charges to your Google Cloud bill you can remove the source topic and the associated subscriptions from your GCP project. Click on the ID (link) of the Pub/Sub topic that you want to delete and choose DELETE from the console top menu. In the Delete topic confirmation box, confirm resource removal by typing delete, then choose DELETE. Repeat this step for any associated topic subscriptions.

13 Repeat steps no. 6 – 12 to enable encryption with Customer-Managed Encryption Keys (CMEKs) for other Pub/Sub topics available within the selected project.

14 Repeat steps no. 2 – 13 for each GCP project available in your Google Cloud account.

01 Before you can set up and manage your Customer-Managed Encryption Keys (CMEKs), you must create a KMS key ring. Run kms keyrings create command (Windows/macOS/Linux) to create a new Cloud KMS key ring in the specified location. If the CMKs created later within this key ring will be used to encrypt/decrypt resources in a given region, select that region as the key ring location:

gcloud kms keyrings create cc-pubsub-key-ring
  --location=global
  --project=cc-dataflow-project-123123
  --format="table(name)"

02 The command output should return the full ID of the newly created key ring:

NAME
projects/cc-dataflow-project-123123/locations/global/keyRings/cc-pubsub-key-ring

03 Run kms keys create command (Windows/macOS/Linux) to create a new Cloud KMS Customer-Managed Encryption Keys (CMEKs) within the KMS key ring created at the previous steps:

gcloud kms keys create cc-pubsub-topic-cmk
  --location=global
  --keyring=cc-pubsub-key-ring
  --purpose=encryption
  --protection-level=software
  --rotation-period=90d
  --next-rotation-time=2020-7-12T10:00:00.0000Z
  --format="table(name)"

04 The command output should return the name of the new Customer-Managed Encryption Key (CMEK):

NAME
projects/cc-dataflow-project-123123/locations/global/keyRings/ccc-pubsub-key-ring/cryptoKeys/cc-pubsub-topic-cmk

05 Run projects add-iam-policy-binding command (Windows/macOS/Linux) to assign the Cloud KMS "CryptoKey Encrypter/Decrypter" role to the necessary service account. Replace <kms-project-id> with the ID of the Google Cloud project where the Customer-Managed Encryption Keys are provisioned, and replace <project-number> with the project number (not the project ID) of the Google Cloud project that is running your Pub/Sub topics:

gcloud projects add-iam-policy-binding <kms-project-id>
  --member serviceAccount:service-<project-number>@gcp-sa-pubsub.iam.gserviceaccount.com
  --role roles/cloudkms.cryptoKeyEncrypterDecrypter

06 The command output should return the updated IAM policy (YAML format):

Updated IAM policy for project <kms-project-id>.
	bindings:
	- members:
		- serviceAccount:service-<project-number>@gcp-sa-pubsub.iam.gserviceaccount.com
		role: roles/cloudkms.cryptoKeyEncrypterDecrypter
	- members:
		- user:admin@cloudconformity.com
		role: roles/owner
	etag: abcdabcdabcd
	version: 1

07 Run pubsub topics list-subscriptions command (Windows/macOS/Linux) using the name of the Pub/Sub topic that you want to re-create as identifier parameter and custom query filters to describe the subscriptions created for the selected topic:

gcloud pubsub topics list-subscriptions cc-dataflow-app-topic
  --format=json

08 The command output should return the requested topic information:

[
	"projects/cc-dataflow-project-123123/subscriptions/cc-dataflow-public-subscription",
	"projects/cc-dataflow-project-123123/subscriptions/cc-dataflow-client-subscription"
]

09 Run pubsub topics create command (Windows/macOS/Linux) to create the new Google Cloud Pub/Sub topic that encrypts published messages with the Customer-Managed Encryption Key (CMEK) created at step no. 3:

gcloud pubsub topics create cc-encrypted-dataflow-topic
  --topic-encryption-key=projects/cc-dataflow-project-123123/locations/global/keyRings/ccc-pubsub-key-ring/cryptoKeys/cc-pubsub-topic-cmk

10 The command output should return the full resource identifier of the newly created topic:

Created topic [projects/cc-dataflow-project-123123/topics/cc-encrypted-dataflow-topic].

11 Run pubsub subscriptions create command (Windows/macOS/Linux) to create the required subscription for the newly created Pub/Sub topic so that subscribers can receive messages published to the new CMK-encrypted topic. Repeat this step to create as many subscriptions as needed:

gcloud pubsub subscriptions create cc-encrypted-dataflow-subscription
  --topic=cc-encrypted-dataflow-topic
  --topic-project=cc-dataflow-project-123123

12 The command output should return the resource ID of the newly created subscription:

Created subscription [projects/cc-dataflow-project-123123/subscriptions/cc-encrypted-dataflow-subscription].

13 If required, update your application to point to the new Pub/Sub topic and subscriptions.

14 To stop adding unnecessary charges to your Google Cloud monthly bill you can remove the source topic from your GCP project. Run pubsub topics delete command (Windows/macOS/Linux) using the ID of the resource that you want to delete (see Audit section part II to identify the source topic), to delete the specified Pub/Sub topic:

gcloud pubsub topics delete cc-dataflow-app-topic

15 The command output should return the resource ID of the deleted Pub/Sub topic:

Deleted topic [projects/massive-woods-273512/topics/cc-dataflow-app-topic]

16 Run pubsub subscriptions delete command (Windows/macOS/Linux) using the name of the topic subscription that you want to delete (see Audit section part II to identify the right resource), to delete the specified subscription:

gcloud pubsub subscriptions delete cc-dataflow-public-subscription

17 The command output should return the resource ID of the deleted topic subscription:

Deleted subscription [projects/cc-dataflow-project-123123/subscriptions/cc-dataflow-public-subscription]

18 Repeat steps no. 7 – 17 to enable encryption with Customer-Managed Encryption Keys (CMEKs) for other Pub/Sub topic provisioned in the selected project.

19 Repeat steps no. 1 – 18 for each GCP project deployed in your Google Cloud account.