Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Underutilized EC2 Instance

Trend Micro Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 750 automated best practice checks.

Risk Level: High (not acceptable risk)
Rule ID: EC2-055

Identify any Amazon EC2 instances that appear to be underutilized and downsize (resize) them to help lower the cost of your AWS bill. By default, an Amazon EC2 instance is considered "underutilized" when matches the following criteria (to declare the instance "underutilized" both conditions must be met):

  • The average CPU utilization has been less than 60% for the last 7 days.
  • The average memory utilization has been less than 60% for the last 7 days. By default, Amazon CloudWatch can't record an EC2 instance memory utilization because the necessary metric cannot be implemented at the hypervisor level, therefore to be able to report the memory utilization using CloudWatch you need to install an agent on the instance that you want to monitor and create a custom metric (we'll name it EC2MemoryUtilization) on the Amazon CloudWatch console. The instructions required for installing the monitoring agent, based on the Operating System (OS) used by the instance, are available at this URL.
Note: You can change the default threshold values for this rule on the Trend Micro Cloud One™ – Conformity console and set your own values for the CPU (percent), memory utilization (percent) and the preferred number of days for each condition to configure a custom underuse level for your Amazon EC2 instances. You can also change the default name for the memory utilization metric (i.e. EC2MemoryUtilization) and use a custom name for this metric. The console also provides information about each EC2 instance marked as underutilized, details such as region, ID, instance type, launch time, operating system and more in order to help you perform the right-sizing analysis.

This rule can help you with the following compliance standards:

  • APRA
  • MAS

For further details on compliance standards supported by Conformity, see here.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Cost
optimisation

Downsizing underutilized Amazon EC2 instances to meet the capacity needs at the lowest cost represents an efficient strategy to reduce your AWS cloud costs. For example, resizing a c4.xlarge-type instance provisioned in the US-East (N. Virginia) region to a c4.large-type instance due to CPU and memory underuse, you can roughly save $72 per month.


Audit

To identify any underused Amazon EC2 instances available within your AWS cloud account, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon EC2 console at https://console.aws.amazon.com/ec2/.

03 In the navigation panel, under Instances, choose Instances.

04 Select the Amazon EC2 instance that you want to examine.

05 Choose the Monitoring tab from the console bottom panel to access the instance monitoring details.

06 On the Monitoring panel, perform the following actions:

  1. Select the CPU utilization (%) graph, click on the 3-dot menu, and choose View in metrics to open the CPU utilization dashboard for the selected instance. On the CPU utilization (%) dashboard, configure the following parameters:
    • Select 1w (1 week) from the time range top-right menu.
    • Select the Graphed metrics tab, set Statistic to Average, and Period to 1 Hour.
  2. Once the monitoring data is loaded, check the instance CPU utilization for the last 7 days. If the average usage (percentage) has been less than 60%, the selected Amazon EC2 instance qualifies as candidate for the underused EC2 instance.

07 Determine the memory utilization for the selected Amazon EC2 instance by reading the EC2MemoryUtilization metric data (or whatever name you have used for your custom metric) reported by the CloudWatch agent installed on the EC2 instance (this conformity rule assumes that the script has been successfully installed and it has returned memory usage data in the past 7 days). To check the Amazon EC2 instance memory utilization, perform the following operations:

  1. Navigate to Amazon Cloudwatch console at https://console.aws.amazon.com/cloudwatch/.
  2. In the navigation panel choose Metrics to access your Cloudwatch metrics.
  3. Choose EC2 from the Metrics section to access the metrics available for the Amazon EC2 resources.
  4. Select Per-Instance Metrics to access the metrics available for the EC2 instances.
  5. Select the EC2MemoryUtilization metric for the Amazon EC2 instance that you want to examine. The EC2MemoryUtilization metric is listed in the Metric Name column.
  6. Select 1w (1 week) from the time range top-right menu to return the data recorded in the past week.
  7. Select Number from the chart type dropdown menu for usage data visualization.
  8. Once the monitoring data is loaded, check the instance memory usage for the last 7 days. If the average usage (percentage) has been less than 60%, the selected Amazon EC2 instance qualifies as candidate for the underused EC2 instance.

08 If all the conditions outlined at step no. 6 and 7 are met, the selected Amazon EC2 instance is considered "underutilized" and can be downsized in order to stop incurring charges for that resource.

09 Repeat steps no. 4 – 8 for each Amazon EC2 instance available within the current AWS region.

10 Change the AWS cloud region from the console navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run describe-instances command (OSX/Linux/UNIX) with custom query filters to list the IDs of the active Amazon EC2 instances available in the selected AWS cloud region:

aws ec2 describe-instances
  --region us-east-1
  --filters Name=instance-state-name,Values=running
  --output table
  --query 'Reservations[*].Instances[*].InstanceId'

02 The command output should return a table with the requested instance identifiers (IDs):

-------------------------
|   DescribeInstances   |
+-----------------------+
|  i-01234abcd1234abcd  |
|  i-0abcdabcdabcdabcd  |
|  i-0abcd1234abcd1234  |
+-----------------------+

03 Run get-metric-statistics command (OSX/Linux/UNIX) to get the utilization data recorded by Amazon CloudWatch for the CPUUtilization metric, representing the CPU usage of the selected Amazon EC2 instance. Change the --start-time (start recording date) and --end-time (stop recording date) parameters values to choose your own time frame for recording the instance CPU usage. Configure the --period parameter value to define the granularity (in seconds) of the returned datapoints. A period can be as short as one minute (60 seconds) or as long as one day (86400 seconds). The following command example returns the average CPU usage of an Amazon EC2 instance identified by the ID i-01234abcd1234abcd, usage data captured over a period of 7 days, using 1-hour period as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
  --region us-east-1
  --metric-name CPUUtilization
  --start-time 2016-10-04T13:16:00
  --end-time 2016-10-11T13:16:00
  --period 3600
  --namespace AWS/EC2
  --statistics Average
  --dimensions Name=InstanceId,Value=i-01234abcd1234abcd

04 The command output should return the CPU usage details requested:

{
	"Datapoints": [
		{
			"Timestamp": "2017-03-04T17:21:00Z",
			"Average": 3.2085,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-04T18:21:00Z",
			"Average": 4.033499999999999995,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-04T19:21:00Z",
			"Average": 1.10425,
			"Unit": "Percent"
		},

		...

		{
			"Timestamp": "2017-03-11T15:21:00Z",
			"Average": 2.030999999999999993,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-11T16:21:00Z",
			"Average": 2.02833333333333333,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-11T17:21:00Z",
			"Average": 3.02783333333333333,
			"Unit": "Percent"
		}
	],
	"Label": "CPUUtilization"
}

If the average CPU usage data returned is less than 60%, the selected Amazon EC2 instance qualifies as candidate for the underused EC2 instance.

05 Determine the Amazon EC2 instance memory usage by querying the EC2MemoryUtilization metric data (or whatever name you have used for your custom metric) reported by the Amazon CloudWatch script installed on the selected EC2 instance (this rule assumes that the script has been successfully installed and it has recorded memory usage data within the past 7 days). To check the instance memory usage reported by your custom Amazon CloudWatch metric, run get-metric-statistics command (OSX/Linux/UNIX) using the metric name as the identifier parameter. The following command example returns the average memory utilization for an Amazon EC2 instance identified by the ID i-01234abcd1234abcd, from the usage data captured by a metric named EC2MemoryUtilization over a period of 7 days, using 1-hour period as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
  --region us-east-1
  --metric-name EC2MemoryUtilization
  --start-time 2016-10-04T13:16:00
  --end-time 2016-10-11T13:16:00
  --period 3600
  --namespace AWS/EC2
  --statistics Average
  --dimensions Name=InstanceId,Value=i-01234abcd1234abcd

06 The command output should return the memory usage data requested:

{
	"Datapoints": [
		{
			"Timestamp": "2017-03-04T17:25:00Z",
			"Average": 5.2085,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-04T18:25:00Z",
			"Average": 6.033499999999999995,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-04T19:25:00Z",
			"Average": 6.10425,
			"Unit": "Percent"
		},

		...

		{
			"Timestamp": "2017-03-11T15:25:00Z",
			"Average": 9.030999999999999993,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-11T16:25:00Z",
			"Average": 9.02833333333333333,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-03-11T17:25:00Z",
			"Average": 9.35783333333333333,
			"Unit": "Percent"
		}
	],
	"Label": "EC2MemoryUtilization"
}

If the average memory utilization data returned is less than 60%, the selected Amazon EC2 instance qualifies as candidate for the underused EC2 instance.

07 If the usage data returned for the steps no. 3 – 6 satisfy all the conditions required by the conformity rule (i.e. instance CPU and memory usage), the selected Amazon EC2 instance is considered "underutilized" and can be downsized in order to stop incurring charges for that resource.

08 Repeat steps no. 3 – 7 for each Amazon EC2 instance available in the selected AWS region.

09 Change the AWS cloud region by updating the **--region**command parameter value and repeat the audit process for other regions.

Remediation / Resolution

Option 1: Downsize (resize) your underused Amazon EC2 instances. To resize any underutilized Amazon EC2 instances running within your AWS cloud account, perform the following actions:

(!) IMPORTANT Note: The following procedure assumes that the Amazon EC2 instances selected for reconfiguration (downsize) are NOT currently used in production or for critical operations.

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon EC2 console at https://console.aws.amazon.com/ec2/.

03 In the navigation panel, under Instances, choose Instances.

04 Select the underused Amazon EC2 instance that you want to reconfigure.

05 Click on the Instance state dropdown button from the console top menu and select Stop instance.

06 In the Stop instance? confirmation box, review the instance details, then choose Stop.

07 Once the instance is stopped (i.e. Instance State is set to stopped), click on the Actions dropdown button from the console top menu, select Instance settings, and choose Change instance type.

08 On the Change instance type configuration page, select the appropriate instance type from the Instance type dropdown list, and choose Apply to resize (downsize) the selected Amazon EC2 instance.

09 Click on the Instance state dropdown button from the console top menu and select Start instance. Once the boot sequence is complete, the EC2 instance status should change from Pending to Running.

10 Repeat steps no. 4 – 9 for each Amazon EC2 instance that you want to downsize, available within the current AWS region.

11 Change the AWS cloud region from the console navigation bar and repeat the remediation process for other regions.

Using AWS CLI

01 Run stop-instances command (OSX/Linux/UNIX) to stop the underused Amazon EC2 instance that you want to reconfigure:

aws ec2 stop-instances
  --region us-east-1
  --instance-ids i-01234abcd1234abcd

02 The output should return the stop-instances command request metadata:

{
	"StoppingInstances": [
		{
			"InstanceId": "i-01234abcd1234abcd",
			"CurrentState": {
				"Code": 64,
				"Name": "stopping"
			},
			"PreviousState": {
				"Code": 16,
				"Name": "running"
			}
		}
	]
}

03 Run modify-instance-attribute command (OSX/Linux/UNIX) to change (downsize) the instance type for your underused Amazon EC2 instance. The following command example changes the instance type for an underused EC2 instance, identified by the ID i-01234abcd1234abcd, from c4.xlarge to c4.large (the command does not produce an output):

aws ec2 modify-instance-attribute
  --region us-east-1
  --instance-id i-01234abcd1234abcd
  --instance-type "{\"Value\": \"c4.large\"}"

04 Run start-instances command (OSX/Linux/UNIX) to restart the reconfigured Amazon EC2 instance (it may take few minutes until the instance enters the running state):

aws ec2 start-instances
  --region us-east-1
  --instance-ids i-01234abcd1234abcd

05 The output should return the start-instances command request metadata:

{
	"StartingInstances": [
		{
			"InstanceId": "i-01234abcd1234abcd",
			"CurrentState": {
				"Code": 0,
				"Name": "pending"
			},
			"PreviousState": {
				"Code": 80,
				"Name": "stopped"
			}
		}
	]
}

06 Repeat steps no. 1 – 5 for each Amazon EC2 instance that you want to downsize, available in the selected AWS region.

07 Change the AWS cloud region by updating the --region command parameter value and repeat the remediation process for other regions.

Option 2: Disable the conformity rule check. If the configuration of your underused Amazon EC2 instance must remain unchanged (some workload scenarios can result in low resource utilization by design), you should turn off the conformity rule check for the specified Amazon EC2 instance from the Trend Micro Cloud One™ – Conformity console.

References

Publication date Mar 13, 2017

Unlock the Remediation Steps


Free 30-day Trial

Automatically audit your configurations with Conformity
and gain access to our cloud security platform.

Confirmity Cloud Platform

No thanks, back to article

You are auditing:

Underutilized EC2 Instance

Risk Level: High