Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Check for Sufficient Log Data Retention Period

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: High (not acceptable risk)

For security, reliability, and compliance purposes, ensure that your Cloud Logging buckets are configured with a data retention period of 365 days or more. A Cloud Logging bucket is a container that stores log data from cloud services such as Compute Engine and App Engine. The retention period represents the number of days to retain log data for a user-defined log bucket and also for the \_Default log bucket.

Security
Reliability

A retention period of 365 days or more should allow you to collect the necessary amount of log data useful to find anomalies and potential security breaches. A sufficient log data retention period also meets regulatory requirements and enables long-term analysis of historical data.


Audit

To determine if your Cloud Logging buckets have a sufficient retention period configured for log data, perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 Navigate to Google Cloud Logging console at https://console.cloud.google.com/logs.

04 In the left navigation panel, under Configure, select Logs Storage to access all the log buckets created for the selected GCP project.

05 Choose the log bucket that you want to examine, excluding the _Required bucket, click on the 3-dot button to open the bucket options menu, and select View bucket details.

06 In the Log bucket details information box, check the Retention period attribute value to determine the number of days to retain data for the selected log bucket. If Retention period is less than 365 days, the selected Cloud Logging bucket does not have a sufficient log data retention period configured.

07 Repeat steps no. 5 and 6 for each log bucket that you want to examine, available for the selected project.

08 Repeat steps no. 2 – 7 for each GCP project deployed in your Google Cloud account.

Using GCP CLI

01 Run projects list command (Windows/macOS/Linux) using custom query filters to list the IDs of all the projects available in your Google Cloud account:

gcloud projects list
  --format="table(projectId)"

02 The command output should return the requested GCP project IDs:

PROJECT_ID
tm-project5-123123
tm-web-prod-123456

03 Run logging buckets list command (Windows/macOS/Linux) using the ID of the GCP project that you want to examine as the identifier parameter and custom query filters to describe the full name of each log bucket available within the selected project, including the system buckets, i.e. _Default and _Required:

gcloud logging buckets list 
  --project=tm-project5-123123 
  --format="json(name)"

04 The command output should return the full name of of each log bucket available. The full name indicates the name of the project and the bucket location:

[
	{
		"name": "projects/tm-project5-123123/locations/global/buckets/tm-server-access-logs-bucket"
	},
	{
		"name": "projects/tm-project5-123123/locations/global/buckets/tm-project5-logs-bucket"
	},
	{
		"name": "projects/tm-project5-123123/locations/global/buckets/_Default"
	},
	{
		"name": "projects/tm-project5-123123/locations/global/buckets/_Required"
	}
]

05 Run logging buckets describe command (Windows/macOS/Linux) using name of the log bucket that you want to examine as the identifier parameter, excluding the _Required bucket, to describe the log data retention period configured for the selected bucket:

gcloud logging buckets describe tm-server-access-logs-bucket 
  --project=tm-project5-123123
  --location=global
  --format="yaml(retentionDays)"

06 The command output should return the retention period configured for the specified log bucket:

retentionDays: 45

Check the retentionDays attribute value to determine the number of days to retain data for the selected log bucket. If the retentionDays value is less than 365 days, the selected Cloud Logging bucket does not have a sufficient log data retention period configured.

07 Repeat steps no. 5 and 6 for each log bucket that you want to examine, available for the selected project.

08 Repeat step no. 3 – 7 for each GCP project available within your Google Cloud account.

Remediation / Resolution

To extend the log data retention period for your Cloud Logging buckets, perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 Navigate to Google Cloud Logging console at https://console.cloud.google.com/logs.

04 In the left navigation panel, under Configure, select Logs Storage to access all the log buckets created for the selected GCP project.

05 Choose the log bucket that you want to configure, excluding the _Required bucket, click on the 3-dot button to open the bucket options menu, and select Edit bucket.

06 Set the Retention Period to 365 days or more (up to 3650 days) and choose Update bucket to apply the configuration changes. The retention rules apply going forward and not retroactively. The log date cannot be recovered after the applicable retention period ends.

07 Repeat steps no. 5 and 6 for each log bucket that you want to configure, available for the selected project.

08 Repeat steps no. 2 – 7 for each GCP project deployed in your Google Cloud account.

Using GCP CLI

01 Run logging buckets describe command (Windows/macOS/Linux) using name of the log bucket that you want to examine as the identifier parameter, excluding the _Required bucket, to describe the log data retention period configured for the selected bucket:

gcloud logging buckets update tm-server-access-logs-bucket
  --project=tm-project5-123123
  --location=global
  --retention-days=365
  --format="yaml(retentionDays)"

02 The command output should return the retention period configured for the specified log bucket:

retentionDays: 365

03 Repeat steps no. 1 and 2 for each log bucket that you want to configure, available for the selected project.

04 Repeat step no. 1 – 3 for each GCP project available within your Google Cloud account.

References

Publication date Mar 26, 2024