Overutilized AWS EC2 Instances

01 Sign in to the AWS Management Console.

02 Navigate to Amazon EC2 console at https://console.aws.amazon.com/ec2/.

03 In the navigation panel, under Instances, choose Instances.

04 Select the Amazon EC2 instance that you want to examine.

05 Choose the Monitoring tab from the console bottom panel to access the instance monitoring details.

06 On the Monitoring panel, perform the following actions:

1. Select the CPU utilization (%) graph, click on the 3-dot menu, and choose View in metrics to open the CPU utilization dashboard for the selected instance. On the CPU utilization (%) dashboard, configure the following parameters:
   • Select 1w (1 week) from the time range top-right menu.
   • Select the Graphed metrics tab, set Statistic to Average, and Period to 1 Hour.
2. Once the monitoring data is loaded, check the instance CPU utilization for the last 7 days. If the average usage (percentage) has been more than 90%, the selected Amazon EC2 instance qualifies as candidate for the overused EC2 instance.

07 Determine the memory utilization for the selected Amazon EC2 instance by reading the EC2MemoryUtilization metric data (or whatever name you have used for your custom metric) reported by the CloudWatch agent installed on the EC2 instance (this conformity rule assumes that the script has been successfully installed and it has returned memory usage data in the past 7 days). To check the Amazon EC2 instance memory utilization, perform the following operations:

1. Navigate to Amazon Cloudwatch console at https://console.aws.amazon.com/cloudwatch/.
2. In the navigation panel choose Metrics to access your Cloudwatch metrics.
3. Choose EC2 from the Metrics section to access the metrics available for the Amazon EC2 resources.
4. Select Per-Instance Metrics to access the metrics available for the EC2 instances.
5. Select the EC2MemoryUtilization metric for the Amazon EC2 instance that you want to examine. The EC2MemoryUtilization metric is listed in the Metric Name column.
6. Select 1w (1 week) from the time range top-right menu to return the data recorded in the past week.
7. Select Number from the chart type dropdown menu for usage data visualization.
8. Once the monitoring data is loaded, check the instance memory usage for the last 7 days. If the average usage (percentage) has been more than 90%, the selected Amazon EC2 instance qualifies as candidate for the overused EC2 instance.

08 If all the conditions outlined at step no. 6 and 7 are met, the selected Amazon EC2 instance is considered "overutilized" and should be upgraded to a better hardware configuration in order to meet your workload needs.

09 Repeat steps no. 4 – 8 for each Amazon EC2 instance available within the current AWS region.

10 Change the AWS cloud region from the console navigation bar and repeat the audit process for other regions.

01 Run describe-instances command (OSX/Linux/UNIX) with custom query filters to list the IDs of all the active Amazon EC2 instances available in the selected AWS cloud region:

aws ec2 describe-instances
  --region us-east-1
  --filters Name=instance-state-name,Values=running
  --output table
  --query 'Reservations[*].Instances[*].InstanceId'

02 The command output should return a table with the requested instance identifiers (IDs):

-------------------------
|   DescribeInstances   |
+-----------------------+
|  i-01234abcd1234abcd  |
|  i-0abcdabcdabcdabcd  |
|  i-0abcd1234abcd1234  |
+-----------------------+

03 Run get-metric-statistics command (OSX/Linux/UNIX) to get the utilization data recorded by Amazon CloudWatch for the CPUUtilization metric, representing the CPU usage of the selected Amazon EC2 instance. Change the --start-time (start recording date) and --end-time (stop recording date) parameters values to choose your own time frame for recording the instance CPU usage. Configure the --period parameter value to define the granularity (in seconds) of the returned datapoints. A period can be as short as one minute (60 seconds) or as long as one day (86400 seconds). The following command example returns the average CPU usage of an Amazon EC2 instance identified by the ID i-01234abcd1234abcd, usage data captured over a period of 7 days, using 1-hour period as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
  --region us-east-1
  --metric-name CPUUtilization
  --start-time 2017-04-21T15:10:00
  --end-time 2017-04-28T15:10:00
  --period 3600
  --namespace AWS/EC2
  --statistics Average
  --dimensions Name=InstanceId,Value=i-01234abcd1234abcd

04 The command output should return the CPU usage details requested:

{
	"Datapoints": [
		{
			"Timestamp": "2017-04-21T14:21:00Z",
			"Average": 153.2085333333333333,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-21T15:21:00Z",
			"Average": 137.03345,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-21T16:21:00Z",
			"Average": 131.4999999999999993,
			"Unit": "Percent"
		},

		...

		{
			"Timestamp": "2017-04-28T12:21:00Z",
			"Average": 312.0365,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-28T13:21:00Z",
			"Average": 290.0283,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-28T14:21:00Z",
			"Average": 227.0278,
			"Unit": "Percent"
		}
	],
	"Label": "CPUUtilization"
}

05 Determine the Amazon EC2 instance memory usage by querying the EC2MemoryUtilization metric data (or whatever name you have used for your custom metric) reported by the Amazon CloudWatch script installed on the selected EC2 instance (this rule assumes that the script has been successfully installed and it has recorded memory usage data within the past 7 days). To check the instance memory usage reported by your custom Amazon CloudWatch metric, run get-metric-statistics command (OSX/Linux/UNIX) using the metric name as the identifier parameter. The following command example returns the average memory utilization for an Amazon EC2 instance identified by the ID i-01234abcd1234abcd, from the usage data captured by a metric named EC2MemoryUtilization over a period of 7 days, using 1-hour period as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
  --region us-east-1
  --metric-name EC2MemoryUtilization
  --start-time 2017-04-21T15:10:00
  --end-time 2017-04-28T15:10:00
  --period 3600
  --namespace AWS/EC2
  --statistics Average
  --dimensions Name=InstanceId,Value=i-01234abcd1234abcd

06 The command output should return the memory usage details requested:

{
	"Datapoints": [
		{
			"Timestamp": "2017-04-21T15:10:00Z",
			"Average": 97.2085,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-21T16:10:00Z",
			"Average": 95.0334,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-21T17:10:00Z",
			"Average": 95.1062,
			"Unit": "Percent"
		},

		...

		{
			"Timestamp": "2017-04-28T13:10:00Z",
			"Average": 98.03999999999999993,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-28T14:10:00Z",
			"Average": 98.02833333333333333,
			"Unit": "Percent"
		},
		{
			"Timestamp": "2017-04-28T15:10:00Z",
			"Average": 93.18783333333333333,
			"Unit": "Percent"
		}
	],
	"Label": "EC2MemoryUtilization"
}

07 If the usage data returned for the steps no. 3 – 6 satisfy all the conditions required by the conformity rule (i.e. average CPU and memory usage above 90%), the selected Amazon EC2 instance is considered "overutilized" and should be upgraded to a better hardware configuration in order to meet your workload needs.

08 Repeat steps no. 3 – 7 for each Amazon EC2 instance available in the selected AWS region.

09 Change the AWS cloud region by updating the --region command parameter value and repeat the audit process for other regions.

01 Sign in to the AWS Management Console.

02 Navigate to Amazon EC2 console at https://console.aws.amazon.com/ec2/.

03 In the navigation panel, under Instances, choose Instances.

04 Select the overused Amazon EC2 instance that you want to reconfigure.

05 Click on the Instance state dropdown button from the console top menu and select Stop instance.

06 In the Stop instance? confirmation box, review the instance details, then choose Stop.

07 Once the instance is stopped (i.e. Instance State is set to stopped), click on the Actions dropdown button from the console top menu, select Instance settings,and chooseChange instance type.

08 On the Change instance typeconfiguration page, select the appropriate instance type from the Instance type dropdown list, and choose Apply to resize (upgrade) the selected Amazon EC2 instance.

09 Click on the Instance state dropdown button from the console top menu and select Start instance. Once the boot sequence is complete, the EC2 instance status should change from Pending to Running.

10 Repeat steps no. 4 – 9 for each Amazon EC2 instance that you want to upgrade (upsize), available within the current AWS region.

11 Change the AWS cloud region from the console navigation bar and repeat the remediation process for other regions.

01 Run stop-instances command (OSX/Linux/UNIX) to stop the overused Amazon EC2 instance that you want to reconfigure:

aws ec2 stop-instances
  --region us-east-1
  --instance-ids i-01234abcd1234abcd

02 The output should return the stop-instances command request metadata:

{
	"StoppingInstances": [
		{
			"InstanceId": "i-01234abcd1234abcd",
			"CurrentState": {
				"Code": 64,
				"Name": "stopping"
			},
			"PreviousState": {
				"Code": 16,
				"Name": "running"
			}
		}
	]
}

03 Run modify-instance-attribute command (OSX/Linux/UNIX) to change (upgrade) the instance type for your overused Amazon EC2 instance. The following command example changes the instance type for an overused EC2 instance, identified by the ID i-01234abcd1234abcd, from c5.xlarge to c5.2xlarge (the command does not produce an output):

aws ec2 modify-instance-attribute
  --region us-east-1
  --instance-id i-01234abcd1234abcd
  --instance-type "{\"Value\": \"c5.2xlarge\"}"

04 Run start-instances command (OSX/Linux/UNIX) to restart the reconfigured Amazon EC2 instance (it may take few minutes until the instance enters the running state):

aws ec2 start-instances
  --region us-east-1
  --instance-ids i-01234abcd1234abcd

05 The output should return the start-instances command request metadata:

{
	"StartingInstances": [
		{
			"InstanceId": "i-01234abcd1234abcd",
			"CurrentState": {
				"Code": 0,
				"Name": "pending"
			},
			"PreviousState": {
				"Code": 80,
				"Name": "stopped"
			}
		}
	]
}

06 Repeat steps no. 1 – 5 for each Amazon EC2 instance that you want to upgrade (upsize), available in the selected AWS region.

07 Change the AWS cloud region by updating the --region command parameter value and repeat the remediation process for other regions.