OpenSearch Domain In VPC

Risk Level: Medium (should be achieved)

Rule ID: ES-010

Ensure that your Amazon OpenSearch domains (clusters) are accessible only from Virtual Private Clouds (VPCs) for better flexibility and control over the cluster access using security groups, as this feature lets you keep all the traffic between your OpenSearch domains and VPC within the AWS network instead of going over the Internet.

This rule can help you with the following compliance standards:

PCI
APRA
MAS

For further details on compliance standards supported by Conformity, see here.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Security

Amazon OpenSearch domains that reside within a VPC have an extra layer of security when compared to OpenSearch domains that use public endpoints. Launching an Amazon OpenSearch cluster within a VPC network enables secure communication between the cluster (domain) and other AWS services without the need for an Internet Gateway, a NAT device or a VPN connection, and all the traffic remains secure within the AWS cloud.

Audit

To determine the access endpoint configuration implemented for your Amazon OpenSearch domains, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.

03 In the main navigation panel, under Dashboard, select Domains.

04 Select the OpenSearch domain that you want to examine and check the configuration value available in the Endpoint column. If the Endpoint is set to Internet, the selected Amazon OpenSearch domain is publicly accessible and does not reside within a Virtual Private Cloud (VPC).

05 Repeat step no. 4 for each Amazon OpenSearch cluster available within the current AWS region.

06 Change the AWS cloud region from the navigation bar and repeat the Audit process for other regions.

Using AWS CLI

01 Run list-domain-names command (OSX/Linux/UNIX) to list the name of each Amazon OpenSearch domain (cluster) available in the selected AWS region:

aws es list-domain-names
  --region us-east-1
  --query 'DomainNames[*].DomainName'

02 The command output should return the identifier (name) of each OpenSearch domain provisioned in the selected region:

[
    "trendmicro",
    "cloudconformity"
]

03 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the Amazon OpenSearch domain that you want to examine as the identifier parameter and custom query filters to describe the access endpoint URL generated for the selected domain:

aws es describe-elasticsearch-domain
  --region us-east-1
  --domain-name trendmicro
  --query 'DomainStatus.Endpoint'

04 The command output should return the requested endpoint URL or null if the selected OpenSearch domain is currently associated with a VPC:

"search-trendmicro-abcd1234abcd1234abcd.us-east-1.es.amazonaws.com"

If the describe-elasticsearch-domain command output returns a public endpoint URL, as shown in the output example above, the selected Amazon OpenSearch domain is publicly accessible and does not reside within a Virtual Private Cloud (VPC).

05 Repeat steps no. 3 and 4 for each Amazon OpenSearch domain available in the selected AWS region.

06 Change the AWS cloud region by updating the --region command parameter value and repeat the Audit process for other regions.

Remediation / Resolution

To migrate your Amazon OpenSearch domain from public access to VPC-based access (recommended), you must re-create the domain with the necessary network access configuration. To relaunch your Amazon OpenSearch domain within a VPC and move the existing data, perform the following actions:

Using AWS CloudFormation

01 CloudFormation template (JSON):

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Enable and Configure VPC-Based Access",
	"Resources": {
		"OpenSearchDomain": {
			"Type":"AWS::OpenSearchService::Domain",
			"Properties": {
				"DomainName": "cc-opensearch-domain",
				"EngineVersion": "OpenSearch_1.1",
				"ClusterConfig": {
					"InstanceType": "t3.small.search",
					"InstanceCount": "2"
				},
				"EBSOptions": {
					"EBSEnabled": true,
					"VolumeType": "gp2",
					"VolumeSize": "50"
				},
				"VPCOptions": {
					"SubnetIds": ["subnet-abcd1234", "subnet-1234abcd"],
					"SecurityGroupIds": ["sg-abcd1234abcd1234"]
				},
				"AccessPolicies": {
					"Version":"2012-10-17",
					"Statement":[
						{
							"Effect": "Deny",
							"Principal": {
								"AWS": "*"
							},
							"Action":"es:*",
							"Resource": "arn:aws:es:us-east-1:123456789012:domain/cc-opensearch-domain/*"
						}
					]
				}
			}
		}
	}
}

02 CloudFormation template (YAML):

AWSTemplateFormatVersion: '2010-09-09'
	Description: Enable and Configure VPC-Based Access
	Resources:
		OpenSearchDomain:
		Type: AWS::OpenSearchService::Domain
		Properties:
			DomainName: cc-opensearch-domain
			EngineVersion: OpenSearch_1.1
			ClusterConfig:
			InstanceType: t3.small.search
			InstanceCount: '2'
			EBSOptions:
			EBSEnabled: true
			VolumeType: gp2
			VolumeSize: '50'
			VPCOptions:
			SubnetIds:
				- subnet-abcd1234
				- subnet-1234abcd
			SecurityGroupIds:
				- sg-abcd1234abcd1234
			AccessPolicies:
			Version: '2012-10-17'
			Statement:
				- Effect: Deny
				Principal:
					AWS: '*'
				Action: es:*
				Resource: arn:aws:es:us-east-1:123456789012:domain/cc-opensearch-domain/*

Using Terraform (AWS Provider)

01 Terraform configuration file (.tf):

terraform {
	required_providers {
		aws = {
			source  = "hashicorp/aws"
			version = "~> 4.0"
		}
	}

	required_version = ">= 0.14.9"
}

provider "aws" {
	region  = "us-east-1"
}

resource "aws_opensearch_domain" "opensearch-domain" {
	domain_name = "cc-opensearch-domain"
	engine_version = "OpenSearch_1.1"

	cluster_config {
		instance_type = "t3.small.search"
		instance_count = 2
	}

	ebs_options {
		ebs_enabled = true
		volume_size = 50
		volume_type = "gp2"
	}

	# Enable and Configure VPC-Based Access 
	vpc_options {
	subnet_ids = ["subnet-abcd1234", "subnet-1234abcd"]
	security_group_ids = ["sg-abcd1234abcd1234"]
	}

	access_policies = <<POLICY
{
	"Version": "2012-10-17",
	"Statement":[
		{
			"Effect": "Deny",
			"Principal": {
				"AWS": "*"
			},
			"Action":"es:*",
			"Resource": "arn:aws:es:us-east-1:123456789012:domain/cc-opensearch-domain/*"
		}
	]
}
POLICY

}

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.

03 In the main navigation panel, under Dashboard, select Domains.

04 Click on the name of the OpenSearch domain that you want to re-create and copy the domain configuration information.

05 Navigate back to the Domains page and choose Create domain to initiate the domain setup process.

06 On the Create domain setup page, perform the following actions:

In the Name section, provide a unique name for your new OpenSearch domain in the Domain name box.
In the Network section, select VPC access (recommended) under Network, to enable VPC-based access for the selected domain. Choose the VPC identifier from the VPC dropdown list, one or more subnets from the Subnets lists, and select one or multiple security groups from the Security groups dropdown list.
Configure the rest of the domain settings using the configuration information copied at step no. 4.
Choose Create to provision your new Amazon OpenSearch domain.

07 Once the new OpenSearch domain is created within the specified Virtual Private Cloud (VPC), upload the data from the source domain to the destination (new) domain.

08 (Optional) You can remove the publicly accessible (source) OpenSearch domain from your AWS account in order to avoid further charges. To delete the unneeded domain, perform the following actions:

In the main navigation panel, under Dashboard, select Domains.
Select the Amazon OpenSearch domain that you want to remove.
Choose Delete from the console top menu to initiate the removal process.
In the Delete domain? confirmation box, type the name of the resource in the required field, then choose Delete to confirm deletion.

09 Repeat steps no. 4 – 8 to enable VPC-based access for other OpenSearch domain available within the current AWS region.

10 Change the AWS cloud region from the navigation bar and repeat the Remediation process for other AWS regions.

Using AWS CLI

01 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the Amazon OpenSearch domain that you want to re-create as the identifier parameter and custom query filters to describe the configuration information available for the selected domain:

aws es describe-elasticsearch-domain
  --region us-east-1
  --domain-name trendmicro

02 The command output should return the configuration details available for the selected OpenSearch domain:

{
    "DomainStatus": {
        "DomainId": "123456789012/trendmicro",
        "DomainName": "trendmicro",
        "ARN": "arn:aws:es:us-east-1:123456789012:domain/trendmicro",
        "Created": true,
        "Deleted": false,
        "Endpoint": "search-trendmicro-abcd1234abcdabcd1234abcd.us-east-1.es.amazonaws.com",
        "Processing": false,
        "UpgradeProcessing": false,
        "ElasticsearchVersion": "7.10",
        "ElasticsearchClusterConfig": {
            "InstanceType": "t3.small.elasticsearch",
            "InstanceCount": 2,
            "DedicatedMasterEnabled": false,
            "ZoneAwarenessEnabled": false,
            "WarmEnabled": false,
            "ColdStorageOptions": {
                "Enabled": false
            }
        },
        "EBSOptions": {
            "EBSEnabled": true,
            "VolumeType": "gp2",
            "VolumeSize": 15
        },
        "AccessPolicies": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"es:*\",\"Resource\":\"arn:aws:es:us-east-1:123456789012:domain/trendmicro/*\"}]}",
        "SnapshotOptions": {},
        "CognitoOptions": {
            "Enabled": false
        },
        "EncryptionAtRestOptions": {
            "Enabled": true,
            "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/12341234-abcd-abcd-abcd-123412341234"
        },
        "NodeToNodeEncryptionOptions": {
            "Enabled": false
        },
        "AdvancedOptions": {
            "indices.fielddata.cache.size": "",
            "override_main_response_version": "false",
            "rest.action.multi.allow_explicit_index": "false"
        },
        "ServiceSoftwareOptions": {
            "CurrentVersion": "R20211203-P2",
            "NewVersion": "",
            "UpdateAvailable": false,
            "Cancellable": false,
            "UpdateStatus": "COMPLETED",
            "Description": "There is no software update available for this domain.",
            "AutomatedUpdateDate": "2021-12-15T22:07:00+00:00",
            "OptionalDeployment": false
        },
        "DomainEndpointOptions": {
            "EnforceHTTPS": false,
            "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07",
            "CustomEndpointEnabled": false
        },
        "AdvancedSecurityOptions": {
            "Enabled": false,
            "InternalUserDatabaseEnabled": false
        },
        "AutoTuneOptions": {
            "State": "ENABLE_IN_PROGRESS"
        }
    }
}

03 Run create-elasticsearch-domain command (OSX/Linux/UNIX) using the configuration information returned at the previous step to relaunch the selected Amazon OpenSearch domain into a Virtual Private Cloud (VPC):

aws es create-elasticsearch-domain
  --region us-east-1
  --domain-name trendmicro-v2
  --elasticsearch-version 7.10
  --elasticsearch-cluster-config InstanceType=t3.small.elasticsearch,InstanceCount=2
  --ebs-options EBSEnabled=true,VolumeType=gp2,VolumeSize=50
  --vpc-options SubnetIds=subnet-abcd1234,subnet-1234abcd,SecurityGroupIds=sg-abcd1234abcd1234
  --access-policies '{"Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Principal": {"AWS": "*" }, "Action":"es:*", "Resource": "arn:aws:es:us-east-1:123456789012:domain/trendmicro/*" } ] }'

04 The command output should return the configuration metadata available for the new OpenSearch domain:

{
    "DomainStatus": {
        "DomainId": "123456789012/trendmicro-v2",
        "DomainName": "trendmicro-v2",
        "ARN": "arn:aws:es:us-east-1:123456789012:domain/trendmicro-v2",
        "Created": true,
        "Deleted": false,
        "Processing": false,
        "UpgradeProcessing": false,
        "ElasticsearchVersion": "7.10",
        "ElasticsearchClusterConfig": {
            "InstanceType": "t3.small.elasticsearch",
            "InstanceCount": 2,
            "DedicatedMasterEnabled": false,
            "ZoneAwarenessEnabled": false,
            "WarmEnabled": false,
            "ColdStorageOptions": {
                "Enabled": false
            }
        },
        "EBSOptions": {
            "EBSEnabled": true,
            "VolumeType": "gp2",
            "VolumeSize": 15
        },
        "AccessPolicies": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"es:*\",\"Resource\":\"arn:aws:es:us-east-1:123456789012:domain/trendmicro/*\"}]}",
        "SnapshotOptions": {},
        "VPCOptions": {
           "SubnetIds": [
               "subnet-abcd1234",
               "subnet-1234abcd"
           ],
           "SecurityGroupIds": [
               "sg-abcd1234abcd1234"
           ]
        },
        "CognitoOptions": {
            "Enabled": false
        },
        "EncryptionAtRestOptions": {
            "Enabled": true,
            "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcd1234-abcd-1234-abcd-1234abcd1234"
        },
        "NodeToNodeEncryptionOptions": {
            "Enabled": false
        },
        "AdvancedOptions": {
            "indices.fielddata.cache.size": "",
            "override_main_response_version": "false",
            "rest.action.multi.allow_explicit_index": "false"
        },
        "ServiceSoftwareOptions": {
            "CurrentVersion": "R20211203-P2",
            "NewVersion": "",
            "UpdateAvailable": false,
            "Cancellable": false,
            "UpdateStatus": "COMPLETED",
            "Description": "There is no software update available for this domain.",
            "AutomatedUpdateDate": "2021-12-15T22:07:00+00:00",
            "OptionalDeployment": false
        },
        "DomainEndpointOptions": {
            "EnforceHTTPS": false,
            "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07",
            "CustomEndpointEnabled": false
        },
        "AdvancedSecurityOptions": {
            "Enabled": false,
            "InternalUserDatabaseEnabled": false
        },
        "AutoTuneOptions": {
            "State": "ENABLE_IN_PROGRESS"
        }
    }
}

05 Once the new OpenSearch domain is launched within the specified Virtual Private Cloud (VPC), upload the data from the source domain to the destination (new) domain.

06 (Optional) You can remove the publicly accessible (source) OpenSearch domain from your AWS account in order to avoid further charges. To shut it down, run delete-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the OpenSearch domain that you want to delete as the identifier parameter:

aws es delete-elasticsearch-domain
  --region us-east-1
  --domain-name trendmicro

07 The command output should return the configuration metadata of the removed OpenSearch domain:

{
    "DomainStatus": {
        "DomainId": "123456789012/trendmicro",
        "DomainName": "trendmicro",
        "ARN": "arn:aws:es:us-east-1:123456789012:domain/trendmicro",
        "Created": true,
        "Deleted": false,
        "Endpoint": "search-trendmicro-abcd1234abcdabcd1234abcd.us-east-1.es.amazonaws.com",
        "Processing": false,
        "UpgradeProcessing": false,
        "ElasticsearchVersion": "7.10",
        "ElasticsearchClusterConfig": {
            "InstanceType": "t3.small.elasticsearch",
            "InstanceCount": 2,
            "DedicatedMasterEnabled": false,
            "ZoneAwarenessEnabled": false,
            "WarmEnabled": false,
            "ColdStorageOptions": {
                "Enabled": false
            }
        },
        "EBSOptions": {
            "EBSEnabled": true,
            "VolumeType": "gp2",
            "VolumeSize": 15
        },
        "AccessPolicies": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"es:*\",\"Resource\":\"arn:aws:es:us-east-1:123456789012:domain/trendmicro/*\"}]}",
        "SnapshotOptions": {},
        "CognitoOptions": {
            "Enabled": false
        },
        "EncryptionAtRestOptions": {
            "Enabled": true,
            "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/12341234-abcd-abcd-abcd-123412341234"
        },
        "NodeToNodeEncryptionOptions": {
            "Enabled": false
        },
        "AdvancedOptions": {
            "indices.fielddata.cache.size": "",
            "override_main_response_version": "false",
            "rest.action.multi.allow_explicit_index": "false"
        },
        "ServiceSoftwareOptions": {
            "CurrentVersion": "R20211203-P2",
            "NewVersion": "",
            "UpdateAvailable": false,
            "Cancellable": false,
            "UpdateStatus": "COMPLETED",
            "Description": "There is no software update available for this domain.",
            "AutomatedUpdateDate": "2021-12-15T22:07:00+00:00",
            "OptionalDeployment": false
        },
        "DomainEndpointOptions": {
            "EnforceHTTPS": false,
            "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07",
            "CustomEndpointEnabled": false
        },
        "AdvancedSecurityOptions": {
            "Enabled": false,
            "InternalUserDatabaseEnabled": false
        },
        "AutoTuneOptions": {
            "State": "ENABLE_IN_PROGRESS"
        }
    }
}

08 Repeat steps no. 1 – 7 to enable VPC-based access for other OpenSearch domain available in the selected AWS region.

09 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 12 to perform the Remediation process for other regions.

References

AWS Command Line Interface (CLI) Documentation
es
list-domain-names
describe-elasticsearch-domain
create-elasticsearch-domain

CloudFormation Documentation
Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) resource type reference

Terraform Documentation
AWS Provider

Publication date Oct 27, 2017

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS CloudFormation

Using Terraform (AWS Provider)

Using AWS Console

Using AWS CLI

References

Related Elasticsearch rules