Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Compute Optimizer EC2 Instance Findings

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)
Rule ID: ComputeOptimizer-001

Check for Compute Optimizer EC2 findings in order to take the necessary actions to optimize under-provisioned and over-provisioned Amazon EC2 instances identified within your AWS cloud account. AWS Compute Optimizer evaluates CPU, network, memory and I/O usage to determine if your Amazon EC2 instances are optimized for your workloads.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Performance
efficiency
Cost
optimisation
Reliability

There are three types of Compute Optimizer EC2 findings:

Under-provisioned – Amazon EC2 instances are considered under-provisioned when at least one instance specification, such as CPU, memory, or network, does not meet the performance requirements of your workload. Under-provisioned EC2 instances can lead to poor application performance.

Over-provisioned – EC2 instances are considered over-provisioned when at least one instance specification, such as CPU, memory, or network, can be sized down while still meeting the performance requirements of your workload. Over-provisioned EC2 instances can lead to unnecessary cost.

Optimized – Amazon EC2 instances are considered optimized when all instance specifications, such as CPU, memory, and network, meet the performance requirements of your workload. Optimized EC2 instances should run your workloads with optimal performance and infrastructure cost.

AWS Compute Optimizer can help you optimize your Amazon EC2 instances by recommending optimal compute resources to reduce costs and improve performance, using machine learning (ML) on historical utilization metrics. You can take advantage of the optimization recommendations provided by Compute Optimizer to reduce costs by up to 25% for over-provisioned instances or increase the overall performance of your workloads for under-provisioned EC2 instances.


Audit

To check your AWS cloud account for Compute Optimizer EC2 instance findings, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to AWS Compute Optimizer console at https://console.aws.amazon.com/compute-optimizer/.

03 In the navigation panel, under AWS Compute Optimizer, choose Dashboard. The Compute Optimizer dashboard provides an overview of the optimization opportunities for your Amazon EC2 resources based on the data that has been collected and analyzed for your current AWS account (or accounts if you are currently signed into the master account of your organization).

04 To clear any preselecting filters, choose Clear filters next to the Regions filter.

05 Click inside the Regions filter, select Region, and choose the AWS region that you want to examine.

06 In the EC2 instances section, check for any Under-provisioned and Over-provisioned optimization findings listed under Findings. If one or more under-provisioned and/or over-provisioned findings were identified, there are Amazon EC2 instances that need to be optimized for cost and performance within the selected AWS region.

07 Change the AWS cloud region using the Regions filter and repeat the audit process for other regions.

Using AWS CLI

01 Run get-recommendation-summaries command (OSX/Linux/UNIX) with custom query filters to return the Amazon EC2 instance optimization findings identified in the selected AWS region (in this case Asia Pacific - Sydney region):

aws compute-optimizer get-recommendation-summaries
	--region ap-southeast-2
	--query 'recommendationSummaries[?(recommendationResourceType==`Ec2Instance`)].summaries[]'

02 The command output should return the number of under-provisioned, over-provisioned, or optimized Amazon EC2 instances, identified in the selected region:

[
    {
        "name": "OPTIMIZED",
        "value": 2.0
    },
    {
        "name": "UNDER_PROVISIONED",
        "value": 2.0
    },
    {
        "name": "OVER_PROVISIONED",
        "value": 5.0
    }
]

If the number of under-provisioned and/or over-provisioned Amazon EC2 instances returned by the get-recommendation-summaries command output is positive, as shown in the example above, there are Amazon EC2 instances that need to be optimized for cost and performance in the selected AWS region.

03 Change the AWS region by updating the --region command parameter value and repeat the entire audit process for other regions.

Remediation / Resolution

To access, review, and implement the Compute Optimizer finding recommendations made for your under-provisioned and/or over-provisioned Amazon EC2 instances, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to AWS Compute Optimizer console at https://console.aws.amazon.com/compute-optimizer/.

03 In the navigation panel, under Recommendations per AWS resource, choose EC2 instances to access the optimization recommendations made by AWS Compute Optimizer service for your under-provisioned and/or over-provisioned Amazon EC2 instances.

04 To clear any preselecting filters, choose Clear filters next to the Regions filter.

05 Click inside the Regions filter, select Region, and choose the AWS region that you want to access.

06 Select the optimization recommendation for the under-provisioned or over-provisioned instance that you want to optimize, and choose View detail.

07 On the selected recommendation page, review each option to identify the optimization recommendation that works best for you. When comparing EC2 instance configurations and determining which one best suits your workload needs, carefully weigh the possible validation efforts against the benefits. Decide whether to optimize for performance improvement, for cost reduction, or for a combination of these two. You can compare up to 3 EC2 instance types, with varying combinations of vCPU, memory, storage, and networking capacity, against the instance type that is currently running. Take performance risk into consideration. Performance risk indicates the amount of effort you might need to spend in order to validate whether the recommended Amazon EC2 instance type meets the performance requirements of your workload.

08 Once you have decided the best alternate configuration for the current instance type, choose Open in EC2 console to access the Amazon EC2 instance that you want to reconfigure. IMPORTANT: The following reconfiguration process assumes that the Amazon EC2 instance selected for optimization is NOT currently used in production or for critical operations.

09 Click on the Instance State dropdown menu, and choose Stop instance.

10 In the Stop Instances confirmation box, review the action details, and choose Stop.

11 Once the instance is stopped (i.e. Instance State is set to stopped), access again the Actions dropdown menu, select Instance Settings, and choose Change Instance Type.

12 In the Change Instance Type configuration box, select the appropriate instance type from the Instance Type dropdown list, and choose Apply to resize the selected EC2 instance.

13 Click on the Instance State dropdown menu, select Start instance.

14 Once the boot sequence is complete, the Amazon EC2 instance status should change from pending to running (this may take few minutes).

15 Repeat steps no. 6 – 14 for each Compute Optimizer finding identified within the current AWS region.

16 Change the AWS cloud region using the Regions filter and repeat the remediation process for other regions.

Using AWS CLI

01 Run get-ec2-instance-recommendations command (OSX/Linux/UNIX) to retrieve the Amazon Resource Name (ARN) of each under-provisioned or over-provisioned EC2 instance identified in the selected AWS region (i.e. Asia Pacific - Sydney region):

aws compute-optimizer get-ec2-instance-recommendations
	--region ap-southeast-2
	--filters name=Finding,values=Underprovisioned,Overprovisioned
	--query 'instanceRecommendations[*].instanceArn'

02 The command output should return the requested instance ARNs:

[
    "arn:aws:ec2:ap-southeast-2:123456789012:instance/i-0abcd1234abcd1234",
    "arn:aws:ec2:ap-southeast-2:123456789012:instance/i-01234abcd1234abcd"
]

03 Run get-ec2-instance-recommendations command (OSX/Linux/UNIX) using the ARN of the under-provisioned/over-provisioned instance that you want to optimize as identifier parameter, to describe the optimization recommendations provided by AWS Compute Optimizer for the selected Amazon EC2 instance:

aws compute-optimizer get-ec2-instance-recommendations
	--region ap-southeast-2
	--instance-arns arn:aws:ec2:ap-southeast-2:123456789012:instance/i-0abcd1234abcd1234
	--query 'instanceRecommendations[*]'

04 The command output should return the optimization recommendations for the selected instance:

[
    {
        "instanceArn": "arn:aws:ec2:ap-southeast-2:123456789012:instance/i-0abcd1234abcd1234",
        "utilizationMetrics": [
            {
                "name": "CPU",
                "value": 50.0,
                "statistic": "MAXIMUM"
            },
            {
                "name": "EBS_READ_OPS_PER_SECOND",
                "value": 662.1,
                "statistic": "MAXIMUM"
            },
            {
                "name": "EBS_WRITE_OPS_PER_SECOND",
                "value": 144.93666666666667,
                "statistic": "MAXIMUM"
            },
            {
                "name": "EBS_READ_BYTES_PER_SECOND",
                "value": 41739361.979166664,
                "statistic": "MAXIMUM"
            },
            {
                "name": "EBS_WRITE_BYTES_PER_SECOND",
                "value": 27068385.416666668,
                "statistic": "MAXIMUM"
            }
        ],
        "recommendationSources": [
            {
                "recommendationSourceType": "Ec2Instance",
                "recommendationSourceArn": "arn:aws:ec2:ap-southeast-2:123456789012:instance/i-0abcd1234abcd1234"
            }
        ],
        "lastRefreshTimestamp": 1605863228.566,
        "recommendationOptions": [
            {
                "performanceRisk": 1.0,
                "projectedUtilizationMetrics": [
                    {
                        "name": "CPU",
                        "value": 89.47368421052632,
                        "statistic": "MAXIMUM"
                    }
                ],
                "instanceType": "r5.xlarge",
                "rank": 1
            },
            {
                "performanceRisk": 3.0,
                "projectedUtilizationMetrics": [
                    {
                        "name": "CPU",
                        "value": 54.83870967741935,
                        "statistic": "MAXIMUM"
                    }
                ],
                "instanceType": "t3.2xlarge",
                "rank": 2
            },
            {
                "performanceRisk": 1.0,
                "projectedUtilizationMetrics": [
                    {
                        "name": "CPU",
                        "value": 50.0,
                        "statistic": "MAXIMUM"
                    }
                ],
                "instanceType": "c5.2xlarge",
                "rank": 3
            }
        ],
        "currentInstanceType": "c5.2xlarge",
        "finding": "OVER_PROVISIONED",
        "lookBackPeriodInDays": 14.0,
        "instanceName": "cc-staging-web-instance",
        "accountId": "123456789012"
    }
]

05 Based on the information returned at the previous step by the get-ec2-instance-recommendations command output, review the optimization recommendations by analyzing the finding properties listed below. Review each recommendation option to identify the instance type configuration that works best for you. When comparing EC2 instance configurations and determining which one best suits your workload needs, weigh the possible validation efforts against the benefits. Decide whether to optimize for performance improvement, for cost reduction, or for a combination of these two:

  1. "finding" - the finding classification returned for the selected EC2 instance (UNDER_PROVISIONED or OVER_PROVISIONED).
  2. "instanceName" - the name of the selected instance.
  3. "currentInstanceType" - the current instance type of the selected Amazon EC2 instance.
  4. "lookBackPeriodInDays" - the number of days for which utilization metrics were analyzed for the selected instance.
  5. "recommendationOptions[].instanceType" - the instance type recommended for the EC2 instance optimization option.
  6. "recommendationOptions[].rank" - the rank of the instance recommendation option. The top recommendation option is ranked as 1.
  7. "recommendationOptions[].projectedUtilizationMetrics" - information that describes the projected utilization metrics of the instance recommendation option.
  8. "recommendationOptions[].performanceRisk" - the performance risk of the instance recommendation option. The performance risk represents the likelihood of the recommended instance type not meeting the performance requirement of your workload. The lowest performance risk is categorized as 0, and the highest as 5.

06 Once you have chosen the best instance type configuration for the selected Amazon EC2 instance, run stop-instances command (OSX/Linux/UNIX) to stop the selected under-provisioned/over-provisioned instance: IMPORTANT: The following reconfiguration process assumes that the Amazon EC2 instance selected for optimization is NOT currently used in production or for critical operations.

aws ec2 stop-instances
	--region ap-southeast-2
	--instance-ids i-0abcd1234abcd1234

07 The output should return the stop-instances command request metadata:

{
    "StoppingInstances": [
        {
            "InstanceId": "i-0abcd1234abcd1234",
            "CurrentState": {
                "Code": 64,
                "Name": "stopping"
            },
            "PreviousState": {
                "Code": 16,
                "Name": "running"
            }
        }
    ]
}

08 Run modify-instance-attribute command (OSX/Linux/UNIX) to resize the selected Amazon EC2 instance to the recommended instance type. The following command example updates the instance type for an over-provisioned EC2 instance, identified by the ID "i-0abcd1234abcd1234", from c5.2xlarge to r5.xlarge (the command does not produce an output):

aws ec2 modify-instance-attribute
	--region ap-southeast-2
	--instance-id i-0abcd1234abcd1234
	--instance-type "{\"Value\": \"r5.xlarge\"}"

09 Run start-instances command (OSX/Linux/UNIX) to restart the optimized Amazon EC2 instance (it may take few minutes until the instance enters the running state):

aws ec2 start-instances
	--region ap-southeast-2
	--instance-ids i-0abcd1234abcd1234

10 The output should return the start-instances command request metadata:

{
    "StartingInstances": [
        {
            "InstanceId": "i-0abcd1234abcd1234",
            "CurrentState": {
                "Code": 0,
                "Name": "pending"
            },
            "PreviousState": {
                "Code": 80,
                "Name": "stopped"
            }
        }
    ]
}

11 Repeat steps no. 3 – 10 for each under-provisioned/over-provisioned EC2 instance (finding) identified in the selected AWS region.

12 Change the AWS region by updating the --region command parameter value and repeat the entire remediation process for other regions.

References

Publication date Dec 14, 2020