Total Number of OpenSearch Cluster Nodes

Risk Level: Medium (should be achieved)

Rule ID: ES-008

Ensure that the number of Amazon OpenSearch cluster nodes (including dedicated master nodes) provisioned in your AWS cloud account has not reached the limit quota established by your organization for the OpenSearch workload deployed. By default, Trend Cloud One™ – Conformity sets a threshold value of 50 for the maximum number of provisioned cluster nodes, however, you can adjust this threshold based on your AWS cloud resource policy upon enabling the rule. Once you define your own threshold for the maximum number of OpenSearch cluster nodes that you need to run across all AWS regions, the Conformity engine will start to continuously scan your account for cluster nodes and when the number of nodes reach the specified threshold you will get notified via the communication channels configured within your Conformity account. If the OpenSearch compute capacity limit quota defined for your AWS account is reached, you can raise an AWS support case where you can request to limit the number of provisioned Amazon OpenSearch cluster nodes.

This rule can help you with the following compliance standards:

APRA
MAS

For further details on compliance standards supported by Conformity, see here.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Security

Cost
optimisation

Monitoring and configuring limits for the number of OpenSearch cluster nodes provisioned within your AWS cloud account will help you to manage better your OpenSearch resources, prevent unexpected charges on your AWS bill, and act fast to mitigate attacks that can use OpenSearch resources. For example, users within your organization can create more OpenSearch cluster nodes than the number established in the company resources policy, exceeding the monthly budget allocated for cloud computing resources. Another example could be a misconfiguration in your Amazon CloudFormation template that can lead to launching more cluster nodes than required. Also, if your AWS account security is compromised and the attackers have the capability to provision a large number of OpenSearch cluster nodes in order to run their malicious data analytics tools, you risk to accrue a lot of AWS charges in a short period of time.

Note: The threshold for the maximum number of OpenSearch cluster nodes per AWS account set for this conformity rule is 50 (default).

Audit

To determine the number of Amazon OpenSearch cluster nodes (data and dedicated master nodes) available within your AWS account, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.

03 In the main navigation panel, under Dashboard, select Domains.

04 Click on the name (link) of the OpenSearch cluster that you want to examine.

05 Select the Cluster configuration tab and check the Number of nodes attribute value listed in the Data nodes and Dedicated master nodes sections to determine the total number of compute nodes provisioned for the selected OpenSearch cluster.

06 Repeat steps no. 4 and 5 to determine the number of nodes provisioned for all other OpenSearch clusters available within the current AWS region.

07 Change the AWS cloud region from the navigation bar and repeat steps no. 3 – 6 for all other regions. If the total number of Amazon OpenSearch cluster nodes (data and dedicated master nodes) provisioned within your AWS account is greater than 50, the threshold limit was exceeded, therefore you must take action and request a limit for the number of cluster nodes that can be provisioned in your account by contacting the AWS Support Center.

Using AWS CLI

01 Run list-domain-names command (OSX/Linux/UNIX) to list the name of each Amazon OpenSearch cluster (domain) available in the selected AWS region:

aws es list-domain-names
  --region us-east-1
  --query 'DomainNames[*].DomainName'

02 The command output should return the identifier (name) of each OpenSearch domain provisioned in the selected region:

[
	"trendmicro",
	"cloudconformity"
]

03 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the Amazon OpenSearch cluster that you want to examine as the identifier parameter and custom query filters to return the number of compute nodes provisioned for the selected cluster:

aws es describe-elasticsearch-domain
  --domain-name trendmicro
  --region us-east-1
  --query 'DomainStatus.ElasticsearchClusterConfig.[{"DataNodeCount":InstanceCount,"DedicatedMasterNodeCount":DedicatedMasterCount}]'

04 The command output should return the number of data nodes and dedicated master nodes:

[
	{
		"DataNodeCount": "4",
		"DedicatedMasterNodeCount": "3"
	}
]

05 Repeat steps no. 3 and 4 to determine the number of nodes provisioned for all other OpenSearch clusters available in the selected AWS region.

06 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 5 for all other regions. If the total number of Amazon OpenSearch cluster nodes (data and dedicated master nodes) provisioned in your AWS account is greater than 50, the threshold limit was exceeded, therefore you must take action and request a limit for the number of cluster nodes that can be provisioned in your cloud account by contacting the AWS Support Center.

Remediation / Resolution

To ensure that the provisioning of Amazon OpenSearch cluster nodes is limited to a certain number, perform the following actions:

Note: Creating a support case to request OpenSearch node count limitations using the AWS Command Line Interface (AWS CLI) is not currently supported.

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to AWS Support Center console at https://console.aws.amazon.com/support/.

03 In the Open support cases section, choose Create case to initiate the request process.

04 On the Create case page, perform the following operations:

Select Account and billing support option.
Select Account from the Type dropdown list.
Select Other Account Issues from the Category dropdown list.
Provide the request subject in the Subject box, e.g. "Limit the creation of Amazon OpenSearch cluster nodes to a total number of 50 nodes per AWS account".
For Description, provide a concise description where you can explain why you need to limit the creation of Amazon OpenSearch cluster nodes to a certain number. This will help the AWS support team to evaluate your request.
For Contact options, choose your preferred correspondence language from the Preferred contact language dropdown list, then select a preferred contact method that AWS support team can use to respond to your request from the Contact methods section.
Choose Submit to send your request to Amazon Web Services. A customer support representative should contact you shortly.

References

AWS Command Line Interface (CLI) Documentation
es
list-domain-names
describe-elasticsearch-domain

Publication date Sep 13, 2017

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS Console

References

Related Elasticsearch rules