Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Backend Buckets Referencing Missing Storage Buckets

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: High (act today)

Ensure that your Cloud CDN backend buckets are referencing existing storage buckets in order to be able to deliver static content efficiently from the nearest edge location to users, reducing latency and improving performance.

Operational
excellence

If your Cloud CDN backend buckets link to deleted (missing) storage buckets, requests trying to access that storage will fail. The CDN won't be able to deliver content, resulting in errors for users trying to access it. To prevent this, ensure that your CDN backend buckets point to existing and active storage buckets.


Audit

To determine if your Cloud CDN backend buckets point to existing and active storage buckets, perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 Navigate to the Cloud CDN console available at https://console.cloud.google.com/net-services/cdn.

04 Click on the name (link) of the Cloud CDN origin that you want to examine.

05 Select the DETAILS tab and check the Origin type attribute value listed under Origin configuration. If the Origin type is set to Backend bucket origin, the origin is a backend bucket resource. Copy the name of the backend bucket listed next to Backend bucket name and continue the Audit process continues with the next step.

06 Navigate to Cloud Load Balancing console available at https://console.cloud.google.com/net-services/loadbalancing.

07 Choose the BACKENDS tab, click in the Filter box, select Name, paste the backend bucket name copied at step no. 5, and press Enter.

08 Click on the name of the resulted backend bucket and copy the name of the storage bucket listed next to Cloud Storage bucket.

09 Navigate to Cloud Storage console at https://console.cloud.google.com/storage and choose Buckets.

10 Click in the Filter box, select Name contains, paste the storage bucket name copied at step no. 8, and press Enter. If the search results do not contain the exact name of the storage bucket associated with your backend bucket, the bucket no longer exists. This means your Cloud CDN backend bucket configuration points to a missing resource.

11 Repeat steps no. 4 - 10 for each Cloud CDN origin that you want to examine, available in the selected GCP project.

12 Repeat steps no. 2 – 11 for each project deployed within your Google Cloud Platform (GCP) account.

Using GCP CLI

01 Run projects list command (Windows/macOS/Linux) with custom query filters to list the ID of each GCP project available within your Google Cloud account:

gcloud projects list 
  --format="table(projectId)"

02 The command output should return the requested GCP project identifier(s):

PROJECT_ID
tm-web-app-project-112233
tm-bigdata-project-123123

03 Run compute url-maps list command (Windows/macOS/Linux) to list the name of the load balancer and the backend resource associated with each CDN origin available in the specified GCP project:

gcloud compute url-maps list 
  --project tm-web-app-project-112233 
  --format="table(name,defaultService)"

04 The command output should return the requested information. If DEFAULT_SERVICE is set to backendBuckets/[resource-name], the CDN origin is associated with a backend bucket resource and the Audit process continues with the next step:

NAME: tm-static-load-balancer
DEFAULT_SERVICE: backendBuckets/tm-backend-bucket

NAME: tm-project5-load-balancer
DEFAULT_SERVICE: backendBuckets/tm-project5-backend

05 Run compute backend-buckets describe command (Windows/macOS/Linux) with custom query filters to describe the name of the storage bucket configured for the specified backend bucket:

gcloud compute backend-buckets describe tm-backend-bucket 
  --project tm-web-app-project-112233
  --format="table(bucketName)"

06 The command output should return the name of the associated storage bucket:

BUCKET_NAME: tm-cdn-content-bucket

07 Run storage buckets describe command (Windows/macOS/Linux) to display the information associated with the specified storage bucket:

gcloud storage buckets describe gs://tm-cdn-content-bucket 
  --project tm-web-app-project-112233

08 The command output should return the requested bucket information:

ERROR: (gcloud.storage.buckets.describe) gs://tm-cdn-content-bucket not found: 404.

If the storage buckets describe command output returns a 404 (not found) error message, as shown in the output example above, the storage bucket no longer exists. Therefore, your Cloud CDN backend bucket configuration points to a missing resource.

09 Repeat steps no. 5 - 8 for each Cloud CDN origin that you want to examine, available in the selected GCP project.

10 Repeat steps no. 3 – 9 for each project created within your Google Cloud Platform (GCP) account.

Remediation / Resolution

To ensure that your Cloud CDN backend buckets link to existing, active storage buckets, perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 Navigate to the Cloud CDN console available at https://console.cloud.google.com/net-services/cdn.

04 Click on the name (link) of the Cloud CDN origin that you want to configure.

05 Select the DETAILS tab and copy the name of the associated load balancer listed next to Associated load balancer, available under Host and path rules.

06 Navigate to Cloud Load Balancing console available at https://console.cloud.google.com/net-services/loadbalancing.

07 Choose the LOAD BALANCERS tab, click in the Filter box, select Name, paste the load balancer name copied at step no. 5, and press Enter.

08 Select the resulted load balancer, click on the 3-dot button to open the resource options menu, and select Edit.

09 On the load balancer configuration page, select Backend configuration, choose the backend bucket that you want to configure, and click on the Edit button (pencil icon) next to the resource name.

10 On the Edit backend bucket page, perform the following actions:

  1. Choose BROWSE next to Cloud Storage bucket to choose a new storage bucket for your backend resource.
  2. On the Select bucket panel, click on the Create new bucket button (plus icon), configure and create your new storage bucket in the Create a bucket setup panel, and choose SELECT to add the new storage bucket to the backend bucket configuration.
  3. Choose UPDATE to apply the configuration changes.

11 Repeat steps no. 4 - 10 for each Cloud CDN origin that you want to configure, available in the selected GCP project.

12 Repeat steps no. 2 – 11 for each project deployed within your Google Cloud Platform (GCP) account.

Using GCP CLI

01 To create the storage bucket that will replace the missing one in your backend bucket configuration, run the storage buckets create command (Windows/macOS/Linux):

gcloud storage buckets create gs://tm-new-cdn-content-bucket 
  --project=tm-web-app-project-112233 
  --default-storage-class=standard 
  --location=us 
  --uniform-bucket-level-access

02 The command output should return the URL of the new storage bucket:

Creating gs://tm-new-cdn-content-bucket1/...

03 Run compute backend-buckets update command (Windows/macOS/Linux) using the name of the backend bucket that you want to configure as the identifier parameter, to replace the missing storage bucket with the one created at the previous step. Use the --gcs-bucket-name command parameter to specify the new storage bucket to serve content from:

gcloud compute backend-buckets update tm-backend-bucket 
  --project=tm-web-app-project-112233 
  --gcs-bucket-name=tm-new-cdn-content-bucket

04 The command output should return the URL of configured backend bucket:

Updated [https://www.googleapis.com/compute/v1/projects/tm-web-app-project-112233/global/backendBuckets/tm-backend-bucket]

05 Repeat steps no. 1 - 4 for each Cloud CDN origin that you want to configure, available in the selected GCP project.

06 Repeat steps no. 1 – 5 for each GCP project created within your Google Cloud Platform (GCP) account.

References

Publication date Mar 26, 2024