Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Enable Blob Storage Lifecycle Management

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)
Rule ID: StorageAccounts-011

Ensure there is a lifecycle management policy configured for your Microsoft Azure Blob Storage data in order to meet compliance requirements when it comes to security and cost optimization. Azure Storage lifecycle management offers a rich, rule-based policy for general purpose and blob storage accounts. Use the lifecycle management policy to transition your Azure cloud data to the appropriate access tiers or expire it at the end of the data's lifecycle.

This rule resolution is part of the Conformity Security & Compliance tool for Azure.

Security
Cost
optimisation

Microsoft Azure Blob Storage service transitions and expires blob data automatically according to the lifecycle management policy that you define at the storage account level. The lifecycle management policy lets you automatically delete blob data at the end of its lifecycle and transition blob data to a cold storage tier (hot to cold, hot to archive or cold to archive) in order to optimize it for security and cost.


Audit

To determine if your Azure Storage accounts have a lifecycle management policy configured, perform the following actions:

Using Azure Portal

01 Sign in to Azure Management Console.

02 Navigate to Azure Storage accounts blade at https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Storage%2FStorageAccounts.

03 On the Storage accounts page, select the subscription that you want to examine from the Subscription filter box.

04 Click on the name of the Azure Storage account that you want to examine.

05 In the navigation panel, under Blob service, choose Lifecycle Management (skip this storage account if it does not exist) to access the lifecycle management policy defined for the selected storage account (if any).

06 On the Lifecycle Management page, select the List view tab and check for any lifecycle management rules configured for the selected account. If there are no lifecycle management rules listed on the List view panel or the status for the available rules is set to Disabled, the blob lifecycle management is not enabled for the data stored within the selected Microsoft Azure Storage account.

07 Repeat steps no. 4 – 6 for each storage account available in the selected Azure subscription.

08 Repeat steps no. 3 – 7 for each subscription created within your Microsoft Azure cloud account.

Using Azure CLI

01 Run storage account list command (Windows/macOS/Linux) using custom query filters to describe the name (and its associated resource group) for each storage account (except kind with 'Storage') available in the current Azure subscription:

az storage account list
    --query '[?kind!=`Storage`].{name:name,resourceGroup:resourceGroup}'

02 The command output should return the requested storage account identifiers:

[
  {
    "name": "1234abcd1234abcd1234abcd",
    "resourceGroup": "cloud-shell-storage-westeurope"
  },
  {
    "name": "abcd1234abcd1234abcd1234",
    "resourceGroup": "cloud-shell-storage-westeurope"
  }
]

03 Run storage account management-policy show command (Windows/macOS/Linux) using the name of the storage account that you want to examine as identifier parameter and custom query filters to describe the names of the active (i.e. enabled) lifecycle management rules configured for the selected Azure Storage account:

az storage account management-policy show
    --account-name "1234abcd1234abcd1234abcd"
    --resource-group cloud-shell-storage-westeurope
    --query 'policy.rules[?(enabled==`true`)].name'

04 The command output should return the name of each lifecycle management rule created for the selected storage account:

No ManagementPolicy found for account 1234abcd1234abcd1234abcd.

If the storage account management-policy show command output returns an empty array (i.e. []), there are no policy rules configured for the selected account, therefore the blob data lifecycle management is not currently enabled. If the command output returns the following error message: "No ManagementPolicy found for account <storage-account-name>.", as shown in the example above, lifecycle management is not enabled for the blob data available in the selected Microsoft Azure Storage account.

05 Repeat step no. 3 and 4 for each storage account available in the selected Azure subscription

06 Repeat steps no. 1 – 5 for each subscription created in your Microsoft Azure cloud account.

Remediation / Resolution

To enable data lifecycle management for your blob storage accounts, define a policy to transition your data to the appropriate access tiers or expire it at the end of the data's lifecycle. To create the required lifecycle management policy, perform the following actions:

Using Azure Portal

01 Sign in to Azure Management Console.

02 Navigate to Azure Storage accounts blade at https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Storage%2FStorageAccounts.

03 On the Storage accounts page, select the subscription that you want to access from the Subscription filter box.

04 Click on the name of the Azure Storage account that you want to reconfigure.

05 In the navigation panel, under Blob service, choose Lifecycle Management and click Add rule to create and configure a new lifecycle management policy.

06 On the Add a rule page, perform the following actions:

  1. In the Rule name box, provide a unique name for your new lifecycle management rule.
  2. To move blobs (objects) to cool storage after a certain number of days after blob last modification, select Move blob to cool storage checkbox and provide the appropriate number of days in the Days after last modification box.
  3. To archive blobs after a certain number of days after blob last modification, select Move blob to archive storage option and provide the appropriate number of days in the Days after last modification box.
  4. To delete blobs after a certain number of days after blob last modification, select Delete blob checkbox and provide the appropriate number of days in the Days after last modification box.
  5. To delete blob snapshots after a certain number of days after snapshot creation, select Delete snapshot option and provide the appropriate number of days in the Days after blob is created box.
  6. Click Next : Filter set > to continue the setup process.
  7. Click Browse and select the blob container or the virtual folder within the container by which to filter. If a blob container is selected, all virtual folders available in that container will be applied to the lifecycle management policy. By default, if no containers/virtual folders are selected, the rule will apply to the entire storage account.
  8. Click Next : Review + add > to review and validate the rule configuration details.
  9. Click Add to add the necessary rule and create your data lifecycle management policy.

07 Repeat steps no. 4 – 6 for each storage account that you want to reconfigure in order to enable data lifecycle management, available in the selected Azure subscription.

08 Repeat steps no. 3 – 7 for each subscription available within your Microsoft Azure cloud account.

Using Azure CLI

01 Define the required lifecycle management policy rule and save the content to a JSON document named lifecycle-mgmt-policy.json. The following rule filters the specified storage account to run the actions on the blob objects that are stored within a blob container named "cc-project5-media-files". The actions configured in the following rule move the blobs in the specified container to cool storage after 30 days after blob last modification, move the objects to archive storage after 90 days after blob last modification, delete the blobs after one year after last modification, and delete blob snapshots after 120 days after snapshot creation:

{
    "rules": [
        {
            "enabled": true,
            "name": "cc-project5-lifecycle-management",
            "type": "Lifecycle",
            "definition": {
                "actions": {
                    "baseBlob": {
                        "tierToCool": {
                            "daysAfterModificationGreaterThan": 30
                        },
                        "tierToArchive": {
                            "daysAfterModificationGreaterThan": 90
                        },
                        "delete": {
                            "daysAfterModificationGreaterThan": 365
                        }
                    },
                    "snapshot": {
                        "delete": {
                            "daysAfterCreationGreaterThan": 120
                        }
                    }
                },
                "filters": {
                    "blobTypes": [
                        "blockBlob"
                    ],
                    "prefixMatch": [
                        "cc-project5-media-files"
                    ]
                }
            }
        }
    ]
}

02 Run storage account management-policy create command (Windows/macOS/Linux) using the name of the storage account that you want to reconfigure as identifier parameter to create a new data lifecycle management policy for the selected storage account, using the policy rules defined at the previous step (i.e. lifecycle-mgmt-policy.json):

az storage account management-policy create
    --account-name "1234abcd1234abcd1234abcd"
    --resource-group cloud-shell-storage-westeurope
    --policy lifecycle-mgmt-policy.json

03 The command output should return the configuration metadata for the newly created data lifecycle management policy:

{
  "id": "/subscriptions/abcdabcd-1234-abcd-1234-abcdabcdabcd/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Storage/storageAccounts/1234abcd1234abcd1234abcd/managementPolicies/default",
  "lastModifiedTime": "2019-12-10T20:23:17.619251+00:00",
  "name": "DefaultManagementPolicy",
  "policy": {
    "rules": [
      {
        "definition": {
          "actions": {
            "baseBlob": {
              "delete": {
                "daysAfterModificationGreaterThan": 365.0
              },
              "tierToArchive": {
                "daysAfterModificationGreaterThan": 90.0
              },
              "tierToCool": {
                "daysAfterModificationGreaterThan": 30.0
              }
            },
            "snapshot": {
              "delete": {
                "daysAfterCreationGreaterThan": 120.0
              }
            }
          },
          "filters": {
            "blobTypes": [
              "blockBlob"
            ],
            "prefixMatch": [
              "cc-project5-media-files"
            ]
          }
        },
        "enabled": true,
        "name": "cc-project5-lifecycle-management"
      }
    ]
  },
  "resourceGroup": "cloud-shell-storage-westeurope",
  "type": "Microsoft.Storage/storageAccounts/managementPolicies"
}

04 Repeat steps no. 1 – 3 for each storage account that you want to reconfigure in order to enable data lifecycle management, available within the current Azure subscription.

05 Repeat steps no. 1 – 4 for each subscription available in your Microsoft Azure cloud account.

References

Publication date Dec 17, 2019