Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Enable Usage and Storage Logs

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)

Ensure that usage and storage logs are enabled for your Google Cloud Storage buckets in order to collect valuable insights into buckets activity, helping monitor access patterns, track costs, detect suspicious behavior, and ensure compliance with security and audit requirements.

Security
Reliability
Cost
optimisation
Performance
efficiency
Operational
excellence

Google Cloud Storage service generates usage and storage logs as downloadable CSV files. Usage logs capture details of all requests made to a specified bucket and are created hourly, while storage logs summarize the bucket's storage consumption for the previous day and are generated daily. Once enable and configured, both usage and storage logs are automatically created and stored as objects in a designated bucket.


Audit

To determine if usage and storage logs are enabled for your Google Cloud Storage buckets, perform the following operations:

Getting the usage and storage log configuration for Cloud Storage buckets is not currently suported by Google Cloud Platform (GCP) console.

Using GCP CLI

01 Run projects list command (Windows/macOS/Linux) with custom output filters to list the ID of each project available in your Google Cloud Platform (GCP) account:

gcloud projects list
	--format="value(projectId)"

02 The command output should return the requested GCP project identifiers (IDs):

cc-project5-123123
cc-ai-project-123123

03 Run storage buckets list command (Windows/macOS/Linux) with custom output filters to describe the identifier (name) of each storage bucket created for the specified GCP project:

gcloud storage buckets list
	--project cc-project5-123123
	--format="value(name)"

04 The command output should return the requested bucket names:

cc-webdata-bucket
cc-dataproc-bucket
cc-cloud-ai-bucket

05 Run storage buckets describe command (Windows/macOS/Linux) with the name of the Cloud Storage bucket that you want to examine as the identifier parameter and custom output filters to describe the usage and storage logging configuration available for the selected bucket:

gcloud storage buckets describe gs://cc-webdata-bucket
	--format="json(logging_config)"

06 The command output should return the requested configuration information:

null

If the storage buckets describe command output returns null, as shown in the example above, usage and storage logs are not enabled for the selected Google Cloud Storage bucket.

07 Repeat steps no. 5 and 6 for each storage bucket created for the selected GCP project.

08 Repeat steps no. 3 – 7 for each GCP project available within your Google Cloud account.

Remediation / Resolution

To enable and configure usage and storage logs for your Google Cloud Storage buckets, perform the following operations:

Enabling usage and storage logs for Cloud Storage buckets is not currently suported by Google Cloud Platform (GCP) console.

Using GCP CLI

01 Run storage buckets create command (Windows/macOS/Linux) to create the storage bucket required to store the log files:

gcloud storage buckets create gs://cc-usage-logs

02 The command output should return the URL of the new bucket:

Creating gs://cc-usage-logs/...

03 Run storage buckets update command (Windows/macOS/Linux) with the name of the Cloud Storage bucket that you want to configure as the identifier parameter, to enable usage and storage logs for the selected storage bucket. Use the --log-bucket parameter to specify the bucket that will receive the log files, created at the previous step. This bucket must exist within the same organization as the bucket being logged. (Optional) Use the --log-object-prefix parameter to specify a prefix for the names of logs generated in the log bucket:

gcloud storage buckets update gs://cc-webdata-bucket
	--log-bucket=cc-usage-logs
	--log-object-prefix=project5-bucket-logs

04 The command output should return the bucket update status:

Updating gs://cc-webdata-bucket/...
Completed 1

05 In addition to enabling logging for your storage bucket, you also need to grant Cloud Storage the roles/storage.objectCreator role. This role gives Cloud Storage, in the form of the group <cloud-storage-analytics@google.com>, permission to create and store your log files as new objects:

gcloud storage buckets add-iam-policy-binding gs://cc-webdata-bucket
	--member=group:cloud-storage-analytics@google.com
	--role=roles/storage.objectCreator

06 The command output should return the modified policy:

bindings:
- members:
	- projectEditor:cc-project5-123123
	- projectOwner:cc-project5-123123
	role: roles/storage.legacyObjectOwner
- members:
	- projectViewer:cc-project5-123123
	role: roles/storage.legacyObjectReader
- members:
	- group:cloud-storage-analytics@google.com
	role: roles/storage.objectCreator
etag: ABCD
kind: storage#policy
resourceId: projects/_/buckets/cc-webdata-bucket
version: 1

07 Repeat steps no. 3 - 6 for each storage bucket that you want to configure, available in the selected GCP project.

08 Repeat steps no. 1 – 7 for each GCP project created within your Google Cloud account.

References

Publication date Dec 17, 2024