Logo of Trend Micro
Use the Knowledge Base AI to help improve your Cloud Posture

Monitor CPU Credit Balance for T2 Instances

Trend Vision One™ provides continuous assurance that gives peace of mind for your cloud infrastructure, delivering over 1100 automated best practice checks.

Risk Level: Medium (should be achieved)

Ensure that the CPU credit balance for all your EC2 T2 burstable instances is being monitored in order to send alert notifications whenever the credit balance becomes dangerously low and can impact your application's performance. Cloud Conformity will send notifications every time the CPU credit balance for a particular EC2 T2 instance goes below a predefined threshold (50 by default).
The AWS CloudWatch metric used to detect T2 instances with low CPU resources is:
CPUCreditBalance - the number of CPU credits available for the EC2 instance to burst beyond its base CPU utilization (Units: Count). CPU credits are available within the credit balance after they are earned and removed from the credit balance after they expire (credits expire 24 hours after they are earned).
Note: You can change the default threshold values for this rule on the Cloud Conformity console and set your own value for the CPU credit balance that is considered low.

Reliability

Detecting EC2 T2 instances with low CPU credit balance is important when these instances are used by latency sensitive applications and can help you take immediate actions in order to maintain an optimal application response time as these can be seriously affected when their instances run out of CPU credits.

Audit

To determine if there are any CloudWatch alarms set up to monitor CPU credit balance for EC2 T2 instances within your AWS account, perform the following:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to CloudWatch dashboard at https://console.aws.amazon.com/cloudwatch/.

03 In the left navigation panel select Alarms.

04 Open the dashboard Show/Hide Columns dialog box by clicking the configuration icon:

configuration icon

then select the Metric Name checkbox.

05 In the Metric Name column, verify each alarm available for the CPUCreditBalance metric. If the CPUCreditBalance metric is not used by any of your existing alarms, the CPU credit balance for EC2 T2 instances is not currently monitored using Amazon CloudWatch alarms.

06 Change the AWS region from the navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run describe-alarms-for-metric command (OSX/Linux/UNIX) to list all AWS CloudWatch alarms that are currently associated with the CPUCreditBalance metric, available in the selected AWS region: 

aws cloudwatch describe-alarms-for-metric
	--region us-east-1
	--metric-name CPUCreditBalance
	--namespace AWS/EC2

02 The command output should return an array with the requested CloudWatch alarm(s) metadata: 

{
    "MetricAlarms": []
}

If the command output returns an empty array, i.e. [ ], for the value of the MetricAlarms property (as shown in the example above), there are no AWS CloudWatch alarms currently set to monitor the CPU credit balance for your EC2 T2 burstable instances.

03 Change the AWS region by updating the --region command parameter value and repeat step no. 1 and 2 to perform the entire audit process for other regions.

Remediation / Resolution

Cloud Conformity is continuously monitoring the CPU credit balance for your EC2 T2 instances and sends alert notifications whenever the credit balance for a particular T2 instance goes below the threshold set within the rule settings (the default threshold is 50 credits). You also have the option to create your custom notification system using AWS CloudWatch and AWS SNS. To create your own notification system, perform the following steps:

Step 1: Create a Simple Notification Service (SNS) topic and the necessary subscription to send notifications whenever the appropriate AWS CloudWatch alarm is triggered:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to SNS dashboard at https://console.aws.amazon.com/sns/v2/.

03 In the navigation panel, select Topics and click the Create new topic button.

04 In the Create new topic dialog box, enter a name and a display name for your topic then click Create Topic.

05 Open the newly created SNS topic configuration page by clicking on its Amazon Resource Name (ARN):

ARN

06 Under Subscription section click Create Subscription.

07 Select Email as subscription protocol from the Protocol dropdown list.

08 In the Endpoint box, enter the email address where you want to receive the CloudWatch alarm notifications then click Create Subscription to create the required subscription.

09 Use your preferred email client application to open the message received from AWS Notifications, then click on the appropriate link to confirm your new email subscription.

Using AWS CLI

01 Run create-topic command (OSX/Linux/UNIX) to create a new SNS topic for sending email notifications whenever the necessary AWS CloudWatch alarm is triggered: 

aws sns create-topic
	--name T2CPUCreditsAlarmSNSTopic

02 The command output should return the Amazon Resource Name (ARN) for the newly created AWS SNS topic: 

{
 "TopicArn": "arn:aws:sns:us-east-1:12345678901:T2CPUCreditsAlarmSNSTopic"
}

03 Run subscribe command (OSX/Linux/UNIX) to send the subscription confirmation message to the notification endpoint (i.e. the email address provided): 

aws sns subscribe
	--topic-arn arn:aws:sns:us-east-1:123456789012:T2CPUCreditsAlarmSNSTopic
	--protocol email
	--notification-endpoint notifyme@cloudconformity.com

04 Run confirm-subscription command (OSX/Linux/UNIX) to confirm the email subscription by validating the token sent to the notification endpoint selected (the command does not produce an output): 

aws sns confirm-subscription
	--topic-arn arn:aws:sns:us-east-1:123456789012:T2CPUCreditsAlarmSNSTopic
	--token d3f9e92f37fb687f5d51e6e241d7700ae02f7124d8268910b858cb4db727ceeb2474bb937929d3bdd7ce5d0cce19325d036bc858d3c217426bcafa9c501a2cace93b83f1dd3797627467553dc438a8c974119496fc3eff026eaa5d14472ded6f9a5c43aec62d83ef5f49109da71e6f832

Step 2: Create the necessary AWS CloudWatch alarm that will fire and send email notifications whenever the CPU credit balance for a specified EC2 T2 instance goes below the threshold set within the conformity rule settings (the default threshold is 50 credits).

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to CloudWatch dashboard at https://console.aws.amazon.com/cloudwatch/.

03 In the left navigation panel select Alarms.

04 Click the Create Alarm button from the CloudWatch dashboard top menu to initiate the process.

05 In the Create Alarm dialog box, perform the following actions:

  1. Select EC2 from the Browse Metrics dropdown list.
  2. Type CPUCreditBalance in the Search Metrics search box and press Enter.
  3. Select the EC2 T2 instance that you want to monitor with AWS CloudWatch from the EC2 > Per-Instance Metrics list, e.g.: Per-Instance Metrics list then click Next to continue the process.
  4. Within the Alarm Threshold section, in the Name and Description fields, enter a unique name and a short description for the new Amazon CloudWatch alarm.
  5. Under Whenever: <Metric Name>, select < (less than) from the is dropdown list and enter 50 as the threshold value (default) in the box next to the dropdown list to trigger the alarm every time the CPU credit balance for the selected EC2 T2 instance goes below the threshold defined.
  6. In the Actions section, click the + Notification button, select State is ALARM from the Whenever this alarm dropdown menu and choose the AWS SNS topic name created at Step 1 from Send notification to dropdown list.
  7. In the Alarm Preview section, select 5 Minutes from the Period dropdown list and choose Standard and Sum from the Statistic category.
  8. Review the CloudWatch alarm configuration details then click Create Alarm. Once the CPU credit balance data is loaded, the State (status) of the new CloudWatch alarm will change from INSUFFICIENT_DATA to OK.

06 Repeat steps no. 3 - 5 to set up CloudWatch alarms for other EC2 T2 instances provisioned within the current region.

07 Change the AWS region from the navigation bar and repeat the remediation/resolution process for other regions.

Using AWS CLI

01 Run put-metric-alarm command (OSX/Linux/UNIX) to create the AWS CloudWatch alarm that will fire every time the CPU credit balance for the selected AWS EC2 T2 instance goes below the threshold set within the rule settings (the default threshold is 50 credits). The following command example creates an Amazon CloudWatch alarm named LowT2CPUCreditsAlarm that send notifications to an AWS SNS topic named T2CPUCreditsAlarmSNSTopic whenever the CPU credit balance for an EC2 instance identified by the ID i-00385d134e8313e92 goes below 50 credits (the command does not return an output): 

aws cloudwatch put-metric-alarm
	--region us-east-1
	--alarm-name LowT2CPUCreditsAlarm
	--alarm-description "Triggered on low CPU credit balance (T2 instances)"
	--metric-name CPUCreditBalance
	--namespace AWS/EC2
	--dimensions "Name=InstanceId,Value=i-00385d134e8313e92"
	--statistic Sum
	--comparison-operator LessThanThreshold
	--evaluation-periods 1
	--period 300
	--threshold 50
	--actions-enabled
	--alarm-actions arn:aws:sns:us-east-1:123456789012:T2CPUCreditsAlarmSNSTopic

02 Repeat step no. 1 to install and configure CloudWatch alarms for other EC2 T2 instances provisioned within the current region.

03 Change the AWS region by updating the --region command parameter value and repeat step no. 1 and 2 to perform the process for other regions.

References

Publication date Jun 12, 2017

Related EC2 rules