Use the Conformity Knowledge Base AI to help improve your Cloud Posture

OpenSearch Reserved Instance Coverage

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)

Ensure that the reservation coverage percentage (i.e. the percentage of instance hours that a reservation covered) available for your Amazon OpenSearch Reserved Instances (RIs) does not fall below a defined threshold. Reserved Instance coverage tracks the number of running instance hours that are covered by reservations, and can be measured over a daily, monthly, quarterly, or yearly cadence. For example, you can monitor your RI coverage either at an aggregate level (e.g. monthly coverage of your entire Amazon OpenSearch RI fleet) or at a more granular level of detail (e.g. monthly coverage of m4.xlarge.elasticsearch instances running within Asia Pacific – Sydney region). The reservation coverage threshold (percentage) and the time period to retrieve data about reservation coverage for, must be defined in the conformity rule settings, on the Trend Cloud One™ – Conformity account console.

This rule can help you work with the AWS Well-Architected Framework.

Cost
optimisation

Reserved Instances (RIs) represent a good strategy to cut down on Amazon OpenSearch costs by taking advantage of AWS billing at a discounted hourly rate. The Reserved Instance coverage report offered by Amazon Cost Management service allows you to discover how much of your overall OpenSearch instance usage is covered by RIs, so that you can make informed decisions about when to purchase or modify a Reserved Instance to ensure maximum reservation coverage. The Cost Management service provides effective recommendations for which reservations to purchase in order to help you reduce your Amazon OpenSearch costs.


Audit

To determine if the reservation coverage (percentage) of your Amazon OpenSearch Reserved Instances fall below your target coverage, perform the following actions:

Using AWS Console

01 Sign in to your Trend Cloud One™ – Conformity account, access OpenSearch Reserved Instance Coverageconformity rule settings and identify the Amazon OpenSearch reservation coverage threshold (percentage) defined for the specified period of time, configured for your AWS cloud account.

01 Sign in to AWS Management Console.

01 Navigate to AWS Cost Management console at https://console.aws.amazon.com/cost-management/home.

01 In the navigation panel, under Reservations, choose Coverage report to access the reservation coverage report provided by Amazon Cost Management service within your AWS account. This report helps you identify opportunities for savings when it comes to Reserved Instances (RIs).

01 On the Coverage report page, perform the following operations:

  1. For FILTERS, select Service, OpenSearch Service, and choose Apply filters to display the RI coverage data for Amazon OpenSearch cluster instances only.
  2. Click on the time period dropdown list to open the date/time picker, and choose the time range to retrieve data about your reservation coverage (daily, monthly, quarterly or yearly) as specified in the conformity rule settings, identified at step no. 1. Once the appropriate time range is selected, choose Apply to show the reservation coverage for that period.
  3. Check the reservation coverage (percentage) displayed in the Average Coverage (hours) panel box, under RI Coverage. If the reservation coverage (percentage) is lower than the target coverage threshold identified at step no. 1, the reservation coverage for your Amazon OpenSearch Reserved Instances (RIs) falls below your target coverage, therefore actions must be taken to ensure maximum coverage.

01 If required, repeat step no. 1 – 5 determine the Amazon OpenSearch reservation coverage for other AWS cloud accounts.

Using AWS CLI

01 Sign in to your Trend Cloud One™ – Conformity account, access OpenSearch Reserved Instance Coverage conformity rule settings and identify the Amazon OpenSearch reservation coverage threshold (percentage) defined for the specified period of time, configured for your AWS cloud account.

02 Define the AWS service for which to return the reservation coverage, i.e. Amazon OpenSearch. Save the following document to a JSON file named aws-service-filter.json:

{
	"Dimensions": { 
		"Key": "SERVICE",
		"Values": [ "Amazon Elasticsearch Service" ]
	}
}

03 Run get-reservation-coverage command (OSX/Linux/UNIX) using the configuration document created at the previous step as value for the --filter parameter and custom query filters to retrieve the reservation coverage for your Amazon OpenSearch Reserved Instances (RIs), for the specified period of time. The --time-period parameter value indicates the start and end dates of the period that you want to retrieve data about reservation coverage for, and must match the time range defined for the target threshold, identified at step no. 1:

aws ce get-reservation-coverage
  --time-period Start=2020-10-05,End=2020-11-05
  --granularity MONTHLY
  --filter file://aws-service-filter.json
  --query 'CoveragesByTime[*].Total.CoverageHours.CoverageHoursPercentage'

04 The command output should return the requested reservation coverage (percentage):

[
	"84"
]

If the reservation coverage (percentage) returned by the get-reservation-coverage command output is lower than the target coverage threshold identified at step no. 1, the reservation coverage for your Amazon OpenSearch Reserved Instances (RIs) falls below your target coverage, therefore actions must be taken to ensure maximum coverage.

05 If required, repeat step no. 1 – 4 determine the Amazon OpenSearch reservation coverage for other AWS cloud accounts.

Remediation / Resolution

To ensure maximum reservation coverage for your Amazon OpenSearch Reserved Instances (RIs), follow the context-aware RI purchase recommendations provided by AWS Cost Management service to help you reduce your AWS costs by performing the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to AWS Cost Management console at https://console.aws.amazon.com/cost-management/home.

03 In the navigation panel, under Reservations, choose Recommendations to access the Reserved Instance (RI) purchase recommendations provided by AWS Cost Management. These recommendations are based on your past usage and indicate opportunities to maximize your cost savings as compared to On-Demand usage.

04 On the Recommendations page, perform the following operations:

  1. For Select recommendation type, choose OpenSearch to show purchase recommendations for Amazon OpenSearch instances only.
  2. For RI Recommendation Parameters, select the required RI attributes and adjust the usage time range for which your purchase recommendations should be calculated:
    • For RI term, select the RI term that you want to use (1-year or 3-year term).
      • For Payment option, choose the best purchase option for you (no upfront, partial upfront, or all upfront).
      • For Based on the past, select the number of previous days that you want AWS to consider when it calculates your recommendations.
    • Once all the required parameters are provided, you can see all the Reserved Instance (RI) purchase recommendations made by AWS Cost Management service in the Purchase Recommendations section. AWS Cost Management recommends the smallest size instance available in an instance class. This makes it easier to purchase a size-flexible Reserved Instances. The service also shows the equal number of normalized units so that you can purchase any instance size that you want. For example, your RI purchase recommendation would be for m4.large.elasticsearch because that is the smallest size instance in the M4 instance class.

05 To adopt the RI recommendations provided by AWS Cost Management service at the previous step and purchase new OpenSearch Reserved Instances (RIs), navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.

06 In the main navigation panel, under Domains, select Reserved Instance leases.

07 Click on the Order Reserved Instances button from the console top menu to initiate the OpenSearch RI purchasing process.

08 On the Purchase reserved instance page, specify the necessary RI offering details based on the RI purchase recommendations provided by AWS at step no 4c:

  1. Provide a unique name, required to identify the reservation, in the Reservation name box.
  2. Select the instance type provided in the RI purchase recommendation from Instance type dropdown list.
  3. Enter the instance size specified for the purchase recommendation listed at step no 4c in the Instance quantity box.
  4. For Term, select the appropriate length of time for the new OpenSearch reservation (1 year or 3 years).
  5. For Payment option (USD), choose the payment plan that you want to use for the new reservation.
  6. Choose Next to continue the process.
  7. Review the Reserved Instance (RI) configuration information and payment plan, then choose Order to purchase your Amazon OpenSearch Reserved Instance.

09 If required, repeat step no. 7 and 8 to purchase more Amazon OpenSearch Reserved Instances as specified in the RI purchase recommendations list provided by AWS Cost Management.

10 If required, repeat step no. 1 – 9 to perform the remediation process for other AWS accounts.

Using AWS Console

01 Run get-reservation-purchase-recommendation command (OSX/Linux/UNIX) with custom output filters to get recommendations for which Amazon OpenSearch Reserved Instances (RIs) to purchase. These recommendations are based on your past usage and indicate opportunities to maximize your cost savings as compared to On-Demand usage. The following command request example retrieves purchase recommendations for All Upfront OpenSearch instances with one-year term, based on the last 30 days of Amazon OpenSearch service usage:

aws ce get-reservation-purchase-recommendation
  --service "Amazon Elasticsearch Service"
  --lookback-period-in-days THIRTY_DAYS
  --term-in-years ONE_YEAR
  --payment-option ALL_UPFRONT
  --query 'Recommendations[*].RecommendationDetails[]'

02 The command output should return the Amazon OpenSearch Reserved Instance purchase recommendation(s) provided by AWS Cost Management service for your AWS account. AWS recommends the smallest size instance in an instance class (family). This makes it easier to purchase a size-flexible Reserved Instance. For example, the following command output describes one RI purchase recommendation for an m4.large.elasticsearch instance:

[
	{
		"AverageUtilization": "100.0000000000",
		"RecurringStandardMonthlyCost": "0.0000000000",
		"MinimumNumberOfInstancesUsedPerHour": "2.0000000000",
		"InstanceDetails": {
			"ESInstanceDetails": {
				"CurrentGeneration": true,
				"InstanceClass": "m4",
				"InstanceSize": "m4.large.elasticsearch",
				"Region": "Asia Pacific (Sydney)",
				"SizeFlexEligible": true
			}
		},
		"EstimatedMonthlySavingsAmount": "102.9686667800",
		"AverageNumberOfInstancesUsedPerHour": "2.0000000000",
		"EstimatedBreakEvenInMonths": "18.0568700706",
		"CurrencyCode": "USD",
		"EstimatedMonthlySavingsPercentage": "21.0000",
		"UpfrontCost": "970.0000000000",
		"MaximumNormalizedUnitsUsedPerHour": "1.0000000000",
		"MaximumNumberOfInstancesUsedPerHour": "2.0000000000",
		"RecommendedNumberOfInstancesToPurchase": "1",
		"EstimatedMonthlyOnDemandCost": "131.2720000000",
		"EstimatedReservationCostForLookbackPeriod": "33.1780820800",
		"RecommendedNormalizedUnitsToPurchase": "1.0",
		"AverageNormalizedUnitsUsedPerHour": "1.0000000000",
		"AccountId": "123456789012",
		"MinimumNormalizedUnitsUsedPerHour": "1.0000000000"
	}
]

03 Run describe-reserved-elasticsearch-instance-offerings command (OSX/Linux/UNIX) to get the pricing details provided by OpenSearch Reserved Instance offerings, based on the RI purchase recommendations provided by AWS Cost Management at the previous step:

aws es describe-reserved-elasticsearch-instance-offerings
  --region ap-southeast-2
  --query 'ReservedElasticsearchInstanceOfferings[?(ElasticsearchInstanceType==`m4.large.elasticsearch`) && (Duration==`31536000`) && (PaymentOption==`ALL_UPFRONT`)]'

04 The command output should return the requested offering information (including the Reserved Instance offering ID – highlighted):

[
	{
		"FixedPrice": 970.0,
		"ReservedElasticsearchInstanceOfferingId": "abcd1234-abcd-1234-abcd-1234abcd1234",
		"RecurringCharges": [
			{
				"RecurringChargeAmount": 0.0,
				"RecurringChargeFrequency": "Hourly"
			}
		],
		"UsagePrice": 0.0,
		"PaymentOption": "ALL_UPFRONT",
		"Duration": 31536000,
		"ElasticsearchInstanceType": "m4.large.elasticsearch",
		"CurrencyCode": "USD"
	}
]

05 Run purchase-reserved-elasticsearch-instance-offering command (OSX/Linux/UNIX) using the ID of the Reserved Instance (RI) offering that you want to access, to purchase Amazon OpenSearch Reserved Instances based on the offering information returned at the previous step:

aws es purchase-reserved-elasticsearch-instance-offering
  --region ap-southeast-2
  --reserved-elasticsearch-instance-offering-id abcd1234-abcd-1234-abcd-1234abcd1234
  --instance-count 1
  --reservation-name cc-opensearch-new-reservation

06 The command output should return the name and the reservation ID of your new Amazon OpenSearch Reserved Instance (RI):

{
	"ReservationName": "cc-opensearch-new-reservation",
	"ReservedElasticsearchInstanceId": "aaaabbbb-aaaa-bbbb-aaaa-aaaabbbbcccc"
}

07 If required, repeat steps no. 3 – 6 to purchase more Amazon OpenSearch Reserved Instances as specified in the RI purchase recommendations list provided by AWS Cost Management.

08 If required, repeat step no. 1 – 7 to perform the entire remediation process for other AWS accounts.

References

Publication date May 31, 2023