Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Cost Anomaly Detection Monitor in Use

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)
Rule ID: CostExplorer-001

Ensure that an Amazon Cost Anomaly Detection monitor is created for your AWS account in order to proactively identify and take action on cost and usage anomalies. A Cost Anomaly Detection monitor tracks each AWS cloud service individually and alerts you for any unexpected cost spikes. You can choose to create your own custom detection monitor or use a pre-built one to receive alert notifications as soon as anomalous spend is detected. You can also use the Cost Anomaly Detection monitor to evaluate specific cost allocation tags, member accounts, and cost categories based on your AWS account structure.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity solution.

Cost
optimisation

Whether there are AWS service misconfigurations, inappropriate or inefficient usage of cloud resources, AWS resources subjected to malicious attacks, or workloads that consume more compute or storage resources as a result of higher traffic and usage, you may want to be informed about unusual spend within your AWS account as soon as possible, and understand what caused the cost overrun. The Amazon Cost Anomaly Detection monitor can automatically track your costs and usage to help you avoid unexpected charges on your AWS bill.


Audit

To determine if there are any Cost Anomaly Detection monitors created within your AWS account, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to AWS Cost Management console at https://console.aws.amazon.com/cost-management/home.

03 In the navigation panel, under AWS Cost Management, choose Cost Anomaly Detection to access the list of anomaly detection cost monitors available in your AWS account. If there are no monitors listed on the Anomaly Detection page, instead a Getting started page is displayed, there are no Cost Anomaly Detection monitors created and configured to send you notification alerts when anomalous spend is detected within your AWS account.

Using AWS CLI

01 Run get-anomaly-monitors command (OSX/Linux/UNIX) with custom query filters to retrieve the cost anomaly monitor definitions created for your AWS cloud account:

aws ce get-anomaly-monitors
  --query 'AnomalyMonitors'

02 The command output should return the requested cost anomaly monitor definitions:

[]

If the get-anomaly-monitors command output returns an empty array, as shown in the example above, there are no Cost Anomaly Detection monitors created and configured to send you notification alerts when anomalous spend is detected within your AWS account.

Remediation / Resolution

To create and configure an Amazon Cost Anomaly Detection monitor to detect cost anomalies at a lower granularity level and identify spend patterns within your AWS account, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to AWS Cost Management console at https://console.aws.amazon.com/cost-management/home.

03 In the navigation panel, under AWS Cost Management, select Cost Anomaly Detection. Click Get started and select Cost monitors tab, click Create monitor to initiate the setup process.

04 On the Create monitor setup page, perform the following operations:

  1. For Monitor types, choose how to monitor your AWS spend by selecting the monitor type that suits your account structure:
    • AWS Services - Recommended – This type of monitor evaluates all AWS services used by your individual AWS account for anomalies. When you add new AWS cloud services, the cost monitor automatically begins to evaluate the new service for anomalies, so you don't have to manually configure your anomaly detection settings. Amazon Web Services (AWS) recommends this monitor type if you don't need to segment your cloud spend by internal organizations or environments.
    • Linked Account – This kind of monitor evaluates the total AWS spend of an individual, or group of member accounts. This is useful if your organization needs to segment spend by team, product, services, or environment, that you define as individual or groups of accounts. For this type, choose the linked accounts that you want to monitor from the Account IDs dropdown list.
    • Cost category – This type of cost anomaly monitor is recommended if you use cost categories to organize and manage your AWS cloud spend. This monitor type is restricted to only one key-value pair. This type will monitor the total costs across the cost category value that you select from the Cost Category dropdown list.
    • Cost allocation tag – This type of cost monitor is similar to Linked Account and is useful if you to need to segment your AWS spend by team, product, services, or environment, defined by cost allocation tags. This monitor type is restricted to one key and accepts multiple values. This monitor type will track the total cost across the tag values specified in the Cost Allocation Tag section.
  2. For Alert threshold, define your alert notification preferences by providing the following information:
    • In the Set your alert threshold box, enter the anomaly impact amount (USD) to start receiving notification alerts. A threshold is not the same as an anomaly. Anomalies are detected using Machine Learning (ML) models. For example, you can set a $0 threshold alert of every anomaly, even if the cost impact is $1.
    • Use the Alerting frequency dropdown list to choose when you want to receive anomaly alerts. You can choose between:
      • Individual alerts – This alert notifies you as soon as a cost anomaly is detected. You might receive multiple alerts throughout a day. To use this type of alert, provide the Amazon Resource Name (ARN) of the required SNS topic in the Amazon Simple Notification Service (Amazon SNS) configuration box. If you don't have an Amazon SNS topic for this purpose, click on the Create or manage existing SNS topics link to create a new SNS topic.
      • Daily summary – This alert notifies you with a daily summary when anomalies are detected. You receive one email notification containing information for multiple anomalies that occurred that day. To use this type of alert, specify the email addresses of your recipients in the Alert recipient box. Provide separate email addresses using commas.
      • Weekly summary – This alert notifies you with a weekly summary when cost anomalies are detected. You receive one email notification containing information for multiple anomalies that occurred that week. To use this type of alert, specify the email addresses of your alert recipients in the Alert recipient configuration box.
  3. For Monitor name, provide a descriptive name that uniquely identifies your new cost monitor.
  4. Click on Create monitor to create your new Amazon Cost Anomaly Detection monitor. Once your monitor is created, Anomaly Detection evaluates your future AWS spend. Based on your defined alerting preferences, you might start receiving notification alerts within 24 hours.

05 To view and analyze any spend anomalies (findings) detected by the newly created monitor within your AWS cloud account, follow the steps outlined in the Cost Anomaly Detection Findings conformity rule.

06 If required, repeat step no. 3 – 5 to create new Cost Anomaly Detection monitors. Do not create monitors that span multiple monitor types as this might lead to evaluating overlapping spends that generate duplicate alerts.

Using AWS CLI

01 Define the new Cost Anomaly Detection monitor configuration as described below, and save the document to a JSON file named cc-detection-monitor-config.json. In the configuration document listed below, the monitor type is the one recommended by Amazon Web Services (AWS), i.e. "AWS Services" pre-built monitor type (default). This type of monitor evaluates all AWS services used by your individual AWS account for anomalies. Choose this monitor type if you don't need to segment your cloud spend by internal organizations or environments:

{
  "MonitorName": "cc-cost-anomaly-detection-monitor",
  "MonitorType": "DIMENSIONAL",
  "MonitorDimension": "SERVICE"
}

02 Run create-anomaly-monitor command (OSX/Linux/UNIX) to create your new Amazon Cost Anomaly Detection monitor using the specifications defined at the previous step (i.e. cc-detection-monitor-config.json file):

aws ce create-anomaly-monitor
  --region us-east-1
  --anomaly-monitor file://cc-detection-monitor-config.json

03 The command output should return the Amazon Resource Name (ARN) of the new monitor:

{
    "MonitorArn": "arn:aws:ce::123456789012:anomalymonitor/abcdabcd-1234-abcd-1234-abcd1234abcd"
}

04 Define the alert notification preferences for your newly created Cost Anomaly Detection monitor. Email subscribers can set an USD threshold and a time frequency for receiving alert notifications. SNS subscribers can set the Amazon Resource Name (ARN) of the required topic. Save the following configuration document to a JSON file named cc-monitor-subscription-config.json. In the subscription example listed below, the anomaly impact amount (USD) to start receiving notification alerts is $500 and the recipient email address is "alert@cloudconformity.com":

{
    "MonitorArnList": [
        "arn:aws:ce::123456789012:anomalymonitor/abcdabcd-1234-abcd-1234-abcd1234abcd"
    ],
    "SubscriptionName": "cc-cost-monitor-subscription",
    "Frequency": "DAILY",
    "Subscribers": [
        {
            "Status": "CONFIRMED",
            "Type": "EMAIL",
            "Address": "alert@cloudconformity.com"
        }
    ],
    "Threshold": 500.0,
    "AccountId": "123456789012"
}

05 Run create-anomaly-subscription command (OSX/Linux/UNIX) to create a subscription for the new Amazon Cost Anomaly Detection monitor, using the specifications defined at the previous step (i.e. cc-monitor-subscription-config.json configuration file):

aws ce create-anomaly-subscription
  --region us-east-1
  --anomaly-subscription file://cc-monitor-subscription-config.json

06 The command output should return the Amazon Resource Name (ARN) of the monitor subscription:

{
    "SubscriptionArn": "arn:aws:ce::123456789012:anomalysubscription/abcd1234-abcd-1234-abcd-1234abcd1234"
}

07 To view and analyze any spend anomalies detected by the newly created monitor within your AWS cloud account, follow the steps outlined in the Cost Anomaly Detection Findings conformity rule.

08 If required, repeat step no. 1 – 7 to create new Amazon Cost Anomaly Detection monitors.

References

Publication date Nov 24, 2020