Logo of Trend Micro
Use the Conformity Knowledge Base AI to help improve your Cloud Posture

AWS CloudFormation Drift Detection

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)
Rule ID: CFM-007

Ensure that your AWS CloudFormation stacks are not drifted from their expected template configuration. A CloudFormation stack is considered to have drifted from its configuration if one or more of its resources have been drifted. A stack resource is considered to have drifted if its actual property values differ from the expected property values specified as stack template parameters. This includes if the property or resource has been deleted. For each resource within the stack that supports drift detection, Amazon CloudFormation compares the actual configuration of the resource with its expected template configuration. Only resource properties explicitly defined in the stack template are checked for drift.

This rule resolution is part of the Conformity solution.

Operational
excellence

With the right permissions, users can implement direct changes to the AWS resources (and their properties) within CloudFormation stacks, without updating the templates. A stack configuration change made outside Amazon CloudFormation can be anything from fixing an AWS Auto Scaling Group parameter to updating an IAM permission or changing an EC2 instance type. Some of these changes may be intentionally made to respond to time-sensitive operational events and some of them may be accidental. Regardless, configuration changes performed outside of CloudFormation are usually problematic and can complicate stack update or deletion operations. Using drift detection feature, you can identify stack resources that have been reconfigured outside Amazon CloudFormation service management. Once drifts are detected, you can then take corrective actions to ensure configuration consistency and successful CloudFormation stack operations.

Audit

To determine if your Amazon CloudFormation stack configurations have been drifted, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to CloudFormation dashboard at https://console.aws.amazon.com/cloudformation/.

03 Select the CloudFormation stack that you want to examine.

04 Click the Stack actions dropdown button from the dashboard top menu and select Detect drift option.

05 Within Detect drift dialog box, click Yes, detect to initiate the detection process. Once the detection process is complete, check the Drift status attribute value. If the attribute value (status) is DRIFTED, the stack configuration has been changed outside CloudFormation management, therefore the selected stack configuration is considered drifted. Click Close to return to the AWS console.

06 Repeat steps no. 3 – 5 to perform drift detection for other Amazon CloudFormation stacks, available in the current region.

07 Change the AWS region from the navigation bar and repeat the audit process for other regions.

Using AWS CLI

Note: Drift detection can be executed only for CloudFormation stacks that are in the following states: "CREATE_COMPLETE", "UPDATE_COMPLETE", "UPDATE_ROLLBACK_COMPLETE" and "UPDATE_ROLLBACK_FAILED".

01 Run list-stacks command (OSX/Linux/UNIX) with status filters to list the names of all CloudFormation stacks available in the selected AWS region: 

aws cloudformation list-stacks
  --region us-east-1
  --stack-status-filter "CREATE_COMPLETE" "UPDATE_COMPLETE" "UPDATE_ROLLBACK_COMPLETE" "UPDATE_ROLLBACK_FAILED"
  --output table
  --query 'StackSummaries[*].StackName'

02 The command output should return a table with the requested stack names: 

-----------------------
|     ListStacks      |
+---------------------+
|  cc-project5-stack  |
|  cc-production-app  |
+---------------------+

03 Run detect-stack-drift command (OSX/Linux/UNIX) using the name of the AWS CloudFormation stack that you want to examine as identifier to perform the drift detection process for the selected stack: 

aws cloudformation detect-stack-drift
  --region us-east-1
  --stack-name cc-project5-stack
  --query 'StackDriftDetectionId'

04 The command output should return the ID associated with the drift detection results: 

"abcdabcd-1234-abcd-1234-abcd1234abcd"

05 Run describe-stack-drift-detection-status command (OSX/Linux/UNIX) using the ID returned at the previous step as input parameter to describe the drift detection operation status returned for the selected CloudFormation stack: 

aws cloudformation describe-stack-drift-detection-status
    --region us-east-1
    --stack-drift-detection-id abcdabcd-1234-abcd-1234-abcd1234abcd
    --query '{DetectionStatus: DetectionStatus, StackDriftStatus: StackDriftStatus}'

06 The command output should return the status of the stack drift detection operation, identified by the DetectionStatus attribute and the status of the stack's actual configuration compared to its expected template configuration, identified by the StackDriftStatus attribute: 

{
    "DetectionStatus": "DETECTION_COMPLETE",
    "StackDriftStatus": "DRIFTED"
}

If the DetectionStatus attribute value is set to "DETECTION_COMPLETE" and theStackDriftStatus value is set to "DRIFTED", as shown in the output example above, the drift detection process has successfully completed for all supported stack resources and the stack configuration has been changed outside Amazon CloudFormation service management, therefore the selected stack configuration is now drifted.

07 Repeat steps no. 3 – 6 to run the drift detection operation for other Amazon CloudFormation stacks, available within the selected region.

08 Perform steps no. 1 – 7 to repeat the entire audit process for the other AWS regions.

Remediation / Resolution

To remediate an AWS CloudFormation stack configuration that has been drifted, you have to update the necessary stack using a template with the expected configuration. Cloud Conformity strongly recommends that you correct any drifted stack configurations using CloudFormation templates, rather than updating the stack resources directly so these can be in accordance with their template definition. To update your Amazon CloudFormation stack configuration, perform the following actions:

Note: As example, this conformity rule demonstrates how to update a drifted CloudFormation stack configuration where an EC2 instance size has been changed (upgraded) outside CloudFormation management.

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to CloudFormation dashboard at https://console.aws.amazon.com/cloudformation/.

03 Select the drifted CloudFormation stack that you want to update (see Audit section part I to identify the right stack).

04 Click the Update button from the dashboard top menu to start the update process.

05 On Specify template step, inside the Prepare template section, choose Use current template to update the stack using the CloudFormation template with the expected configuration.

06 Click the Next button to continue the process.

07 On Specify stack details step, make sure that the stack parameters are set to their expected values. In this case, change the EC2 instance type parameter to the instance type configured directly (i.e. outside CloudFormation management).

08 Click Next until you reach the Review step, then check the entire configuration of the CloudFormation stack before updating it.

09 Click Update to update the selected CloudFormation stack. Once the stack has been successfully updated, its status should change from UPDATE_IN_PROGRESS to UPDATE_COMPLETE.

10 If required, repeat steps no. 3 – 10 to update other drifted Amazon CloudFormation stacks available in the selected region.

11 Change the AWS region from the navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run describe-stack-resource-drifts command (OSX/Linux/UNIX) using the name of the drifted CloudFormation stack as identifier and custom query filters to return the drift information for the resources that have been checked for drift in the selected stack. This includes actual and expected configuration values for resources where Amazon CloudFormation detects configuration drifts. The information returned by describe-stack-resource-drifts command will help you update the necessary stack configuration with the appropriate parameters: 

aws cloudformation describe-stack-resource-drifts
    --region us-east-1
    --stack-name cc-project5-stack
    --query 'StackResourceDrifts[*].{LogicalResourceId: LogicalResourceId, PropertyDifferences: PropertyDifferences}'

02 The command output should return the logical ID, the actual and the expected configuration values (if applicable) for each AWS resource within the drifted CloudFormation stack. The following command output example, describes an EC2 instance, identified by the logical ID "WebServerInstance", that has been reconfigured when its instance size was changed outside CloudFormation service management: 

[
    ...
    {
        "LogicalResourceId": "WebServerInstance",
        "PropertyDifferences": [
            {
                "PropertyPath": "/InstanceType",
                "ActualValue": "m4.xlarge",
                "ExpectedValue": "m4.large",
                "DifferenceType": "NOT_EQUAL"
            }
        ]
    }
    ...
]

03 Run update-stack command (OSX/Linux/UNIX) to update the drifted Amazon CloudFormation stack (see Audit section part II to identify the right stack) in order to include the resource configuration changed outside CloudFormation management within the expected stack configuration (in this case update the stack to change the EC2 instance type parameter to the instance type configured directly, outside CloudFormation): 

aws cloudformation update-stack
    --region us-east-1
    --stack-name cc-project5-stack
    --use-previous-template
    --parameters ParameterKey=InstanceType,ParameterValue=m4.xlarge,UsePreviousValue=false,ResolvedValue=m4.large

04 The command output should return the ID of the updated AWS CloudFormation stack: 

{
   "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/cc-project5-stack/12341234-abcd-abcd-abcd-123412341234"
}

05 If required, repeat steps no. 1 – 4 to update other drifted Amazon CloudFormation stacks available in the selected region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the entire process for other regions.

References

Publication date Nov 21, 2018

Related CloudFormation rules