Redshift Cluster In VPC

Risk Level: Medium (should be achieved)

Rule ID: RS-004

Ensure that all your Amazon Redshift clusters are provisioned within the EC2-VPC platform instead of EC2-Classic platform (outdated) for better flexibility and control over clusters security, advanced traffic routing, and high availability.

This rule can help you with the following compliance standards:

PCI
HIPAA
APRA
MAS
NIST4

For further details on compliance standards supported by Conformity, see here.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Security

Creating and managing Amazon Redshift clusters using the EC2-VPC platform instead of EC2-Classic can bring multiple advantages such as better networking infrastructure (network isolation, cluster subnet groups, and Elastic IP addresses), much more flexible control over access security (network ACLs, VPC security group outbound traffic filtering), and last but not least, access to newer and powerful node types (DS2).

Audit

To determine the type of the EC2 platform (EC2-Classic or EC2-VPC) used to launch your Amazon Redshift clusters, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon Redshift console at https://console.aws.amazon.com/redshiftv2/.

03 In the main navigation panel, under Provisioned clusters dashboard, choose Clusters.

04 Click on the name (link) of the Redshift cluster that you want to examine.

05 Choose the Properties tab to access the cluster network and security configuration details.

06 In the Network and security settings section, check the Virtual private cloud (VPC) attribute value. If the Virtual private cloud (VPC) attribute does not have a value (i.e. a VPC ID), the selected Amazon Redshift cluster is not running within a Virtual Private Cloud (EC2-VPC platform), instead the selected cluster is using the outdated EC2-Classic platform.

07 Repeat steps no. 4 – 6 for each Redshift cluster deployed within the current AWS region.

08 Change the AWS cloud region from the navigation bar and repeat the Audit process for other regions.

Using AWS CLI

01 Run describe-clusters command (OSX/Linux/UNIX) with custom query filters to list the identifier (name) of each Amazon Redshift cluster available in the selected region:

aws redshift describe-clusters
  --region us-east-1
  --output table
  --query 'Clusters[*].ClusterIdentifier'

02 The command output should return a table with the requested cluster names:

-------------------------
|   DescribeClusters    |
+-----------------------+
|  cc-redshift-cluster  |
|  cc-project5-cluster  |
|  cc-analytics-cluster |
+-----------------------+

03 Run describe-clusters command (OSX/Linux/UNIX) using the name of the Amazon Redshift cluster that you want to examine as the identifier parameter and custom query filters to describe the ID of the Amazon Virtual Private Cloud (VPC) that hosts the selected cluster:

aws redshift describe-clusters
  --region us-east-1
  --cluster-identifier cc-redshift-cluster
  --query 'Clusters[*].VpcId'

04 The command output should return the requested VPC network identifier:

[]

If the describe-clusters command output returns an empty array (i.e. []), as shown in the output example above, the selected Amazon Redshift cluster is not running within a Virtual Private Cloud (EC2-VPC platform), instead the selected cluster is using the outdated EC2-Classic platform where Redshift clusters run inside a single, flat network that is shared with other AWS customers.

05 Repeat steps no. 3 and 4 for each Redshift cluster provisioned in the selected AWS region.

06 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the Audit process for other AWS regions.

Remediation / Resolution

To migrate your EC2-Classic Redshift clusters to a Virtual Private Cloud (VPC), you must re-create your clusters within a VPC network. To relaunch the required Amazon Redshift clusters, perform the following actions:

Using AWS CloudFormation

01 CloudFormation template (JSON):

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Deploy Redshift Cluster to EC2-VPC Platform",
	"Parameters": {
		"ClusterName": {
			"Default": "cc-redshift-cluster",
			"Description": "Redshift cluster name",
			"Type": "String",
			"MinLength": "1",
			"MaxLength": "63",
			"AllowedPattern": "^[0-9a-zA-Z-/]*$",
			"ConstraintDescription": "Must begin with a letter and must not end with a hyphen or contain two consecutive hyphens."
		},
		"ClusterNodeType": {
			"Default": "dc2.large",
			"Description": "Cluster node type",
			"Type": "String",
			"ConstraintDescription": "Must provide a valid cluster node type."
		},
		"DBName": {
			"Description": "Cluster database name",
			"Type": "String",
			"MinLength": "1",
			"MaxLength": "64",
			"AllowedPattern": "[a-zA-Z][a-zA-Z0-9]*",
			"ConstraintDescription": "Must begin with a letter and contain only alphanumeric characters."
		},
		"DBUsername": {
			"Description": "Master username for cluster database access",
			"Type": "String",
			"MinLength": "1",
			"MaxLength": "16",
			"AllowedPattern": "[a-zA-Z][a-zA-Z0-9]*",
			"ConstraintDescription": "Must begin with a letter and contain only alphanumeric characters."
		},
		"DBPassword": {
			"NoEcho": "true",
			"Description": "Password for cluster database access",
			"Type": "String",
			"MinLength": "8",
			"MaxLength": "41",
			"AllowedPattern": "[a-zA-Z0-9]*",
			"ConstraintDescription": "Must contain only alphanumeric characters."
		}
	},
	"Resources": {
		"ClusterSubnetGroup": {
			"Type": "AWS::Redshift::ClusterSubnetGroup",
			"Properties": {
				"Description": "cc-custom-subnet-group",
				"SubnetIds": [
					"subnet-01234abcd1234abcd",
					"subnet-0abcd1234abcd1234"
				]
			}
		},
		"RedshiftCluster": {
			"Type": "AWS::Redshift::Cluster",
			"Properties": {
				"ClusterIdentifier": {
					"Ref": "ClusterName"
				},
				"DBName": {
					"Ref": "DBName"
				},
				"MasterUsername": {
					"Ref": "DBUsername"
				},
				"MasterUserPassword": {
					"Ref": "DBPassword"
				},
				"NodeType": {
					"Ref": "ClusterNodeType"
				},
				"ClusterType": "single-node",
				"ClusterSubnetGroupName": {
					"Ref": "ClusterSubnetGroup"
				},
				"VpcSecurityGroupIds": [
					"sg-0abcd1234abcd1234",
					"sg-01234abcd1234abcd"
				],
				"AvailabilityZone": "us-east-1e"
			}
		}
	}
}

02 CloudFormation template (YAML):

AWSTemplateFormatVersion: '2010-09-09'
	Description: Deploy Redshift Cluster to EC2-VPC Platform
	Parameters:
	ClusterName:
		Default: cc-redshift-cluster
		Description: Redshift cluster name
		Type: String
		MinLength: '1'
		MaxLength: '63'
		AllowedPattern: ^[0-9a-zA-Z-/]*$
		ConstraintDescription: Must begin with a letter and must not end with a hyphen
		or contain two consecutive hyphens.
	ClusterNodeType:
		Default: dc2.large
		Description: Cluster node type
		Type: String
		ConstraintDescription: Must provide a valid cluster node type.
	DBName:
		Description: Cluster database name
		Type: String
		MinLength: '1'
		MaxLength: '64'
		AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
		ConstraintDescription: Must begin with a letter and contain only alphanumeric
		characters.
	DBUsername:
		Description: Master username for cluster database access
		Type: String
		MinLength: '1'
		MaxLength: '16'
		AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
		ConstraintDescription: Must begin with a letter and contain only alphanumeric
		characters.
	DBPassword:
		NoEcho: 'true'
		Description: Password for cluster database access
		Type: String
		MinLength: '8'
		MaxLength: '41'
		AllowedPattern: '[a-zA-Z0-9]*'
		ConstraintDescription: Must contain only alphanumeric characters.
	Resources:
	ClusterSubnetGroup:
		Type: AWS::Redshift::ClusterSubnetGroup
		Properties:
		Description: cc-custom-subnet-group
		SubnetIds:
			- subnet-01234abcd1234abcd
			- subnet-0abcd1234abcd1234
	RedshiftCluster:
		Type: AWS::Redshift::Cluster
		Properties:
		ClusterIdentifier: !Ref 'ClusterName'
		DBName: !Ref 'DBName'
		MasterUsername: !Ref 'DBUsername'
		MasterUserPassword: !Ref 'DBPassword'
		NodeType: !Ref 'ClusterNodeType'
		ClusterType: single-node
		ClusterSubnetGroupName: !Ref 'ClusterSubnetGroup'
		VpcSecurityGroupIds:
			- sg-0abcd1234abcd1234
			- sg-01234abcd1234abcd
		AvailabilityZone: us-east-1e

Using Terraform (AWS Provider)

01 Terraform configuration file (.tf):

terraform {
	required_providers {
		aws = {
			source  = "hashicorp/aws"
			version = "~> 4.0"
		}
	}

	required_version = ">= 0.14.9"
}

provider "aws" {
	profile = "default"
	region  = "us-east-1"
}

resource "aws_redshift_subnet_group" "redshift-custom-subnet-group" {
	name       = "cc-custom-subnet-group"
	subnet_ids = ["subnet-01234abcd1234abcd","subnet-0abcd1234abcd1234"]
}

resource "aws_redshift_cluster" "redshift-database-cluster" {
	cluster_identifier        = "cc-redshift-prod-cluster"
	node_type                 = "dc2.large"
	database_name             = "clusterdb"
	master_username           = "masterdbuser"
	master_password           = "masteruserpwd"
	cluster_type              = "single-node"

	# Deploy Redshift Cluster to EC2-VPC Platform
	cluster_subnet_group_name = aws_redshift_subnet_group.redshift-custom-subnet-group.name
	vpc_security_group_ids    = [ "sg-0abcd1234abcd1234", "sg-01234abcd1234abcd" ]
	availability_zone         = "us-east-1e"
}

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon Redshift console at https://console.aws.amazon.com/redshiftv2.

03 In the main navigation panel, under Provisioned clusters dashboard, choose Clusters.

04 Click on the name of the Redshift cluster that you want to re-create.

05 Choose Actions from the console top menu and select Create snapshot underBackup and disaster recovery.

06 In the Create snapshot dialog box, provide a unique name for your Redshift cluster snapshot in the Snapshot identifier box, specify how long do you want to retain your snapshot, then choose Create snapshot to take the snapshot.

07 In the main navigation panel, under Reserved nodes, choose Snapshots.

08 Select the cluster snapshot created at step no. 6, choose Restore from snapshot, and select Restore to provisioned cluster.

09 On the Restore snapshot <snapshot-name> page, perform the following operations:

Provide a unique name for your new Amazon Redshift cluster in the Cluster identifier box.
In the Additional configurations section, perform the following operations:
- Toggle Use defaults to turn off the default configuration settings.
- Choose Network and security to access the cluster network and security configuration settings.
- For Virtual private cloud (VPC), choose the Virtual Private Cloud (VPC) where the new Redshift cluster will be launched.
- For VPC security groups, select one or more security groups. These security groups define which subnets and IP ranges the cluster can use in the selected VPC.
- For Cluster subnet group, choose the Amazon Redshift subnet group to launch the new cluster in.
- For Availability Zone, specify the Availability Zone (AZ) that you want the cluster to be created in. Otherwise, Amazon Redshift chooses an Availability Zone for you.
- For Enhanced VPC routing, choose whether to enable the Enhanced VPC Routing feature.
- For Publicly accessible, configure the access to your new Redshift cluster.
Ensure that the rest of the configuration settings (cluster size, cluster permissions, backup and security settings) are correct.
Choose Restore cluster from snapshot to create the new Amazon Redshift cluster.

10 As soon as the restoration process is complete, update your application configuration to point to the new cluster endpoint.

11 (Optional) To remove the source Redshift cluster from your AWS cloud account in order to eliminate any unnecessary expenses, perform the following actions:

In the main navigation panel, under Provisioned clusters dashboard, select Clusters.
Select the EC2-Classic Redshift cluster that you want to remove, choose Actions, and select Delete under Manage cluster.
In the Delete <cluster-name> dialog box, select Create final snapshot (optional), enter a unique name for this snapshot in the Final snapshot identifierbox, enter delete in the required field, and choose Delete cluster.

12 Repeat steps no. 4 – 11 for each Redshift cluster that you want to re-create, available within the current AWS region.

13 Change the AWS cloud region from the navigation bar and repeat the Remediation process for other regions.

Using AWS CLI

01 Run describe-clusters command (OSX/Linux/UNIX) to describe the configuration information available for the Amazon Redshift cluster that you want to reconfigure:

aws redshift describe-clusters
  --region us-east-1
  --cluster-identifier cc-redshift-cluster
  --query 'Clusters'

02 The command output should return the requested configuration information. This information will be useful for creating the new Amazon Redshift cluster:

[
	{
		"ClusterIdentifier": "cc-redshift-cluster",
		"NodeType": "dc2.large",
		"ClusterStatus": "available",
		"ClusterAvailabilityStatus": "Available",
		"MasterUsername": "dbauser",
		"DBName": "redshiftdb",
		"Endpoint": {
			"Address": "cc-redshift-cluster.abcd1234abcd.us-east-1.redshift.amazonaws.com",
			"Port": 5439
		},
		"AutomatedSnapshotRetentionPeriod": 1,
		"ManualSnapshotRetentionPeriod": -1,
		"PreferredMaintenanceWindow": "wed:08:00-wed:08:30",
		"PendingModifiedValues": {},
		"ClusterVersion": "1.0",
		"NumberOfNodes": 1,
		"PubliclyAccessible": false,
		"Encrypted": false,
		"Tags": [],
		"IamRoles": [],
		"MaintenanceTrackName": "current",
		"ClusterNamespaceArn": "arn:aws:redshift:us-east-1:123456789012:namespace:abcd1234-abcd-1234-abcd-1234abcd1234",
		"TotalStorageCapacityInMegaBytes": 400000
	}
]

03 Run create-cluster-snapshot command (OSX/Linux/UNIX) to create a manual snapshot for the Amazon Redshift cluster that you want to reconfigure:

aws redshift create-cluster-snapshot
  --region us-east-1
  --cluster-identifier cc-redshift-cluster
  --snapshot-identifier cc-redshift-cluster-snapshot

04 The command output should return the metadata available for the cluster snapshot:

{
	"Snapshot": {
		"SnapshotIdentifier": "cc-redshift-cluster-snapshot",
		"ClusterIdentifier": "cc-redshift-cluster",
		"SnapshotCreateTime": "2021-11-20T10:33:45.190000+00:00",
		"Status": "creating",
		"Port": 5439,
		"MasterUsername": "dbauser",
		"ClusterVersion": "1.0",
		"SnapshotType": "manual",
		"NodeType": "dc2.large",
		"NumberOfNodes": 1,
		"DBName": "redshiftdb",
		"Encrypted": false,
		"EncryptedWithHSM": false,
		"OwnerAccount": "123456789012",
		"TotalBackupSizeInMegaBytes": -1.0,
		"ActualIncrementalBackupSizeInMegaBytes": -1.0,
		"BackupProgressInMegaBytes": 0.0,
		"CurrentBackupRateInMegaBytesPerSecond": 0.0,
		"EstimatedSecondsToCompletion": -1,
		"ElapsedTimeInSeconds": 0,
		"Tags": [],
		"MaintenanceTrackName": "current",
		"ManualSnapshotRetentionPeriod": -1
	}
}

05 Run restore-from-cluster-snapshot command (OSX/Linux/UNIX) to create a new Amazon Redshift cluster from the cluster snapshot created at the previous steps, using the configuration information returned at step no. 2 and the network details of the Virtual Private Cloud (VPC) where the new Amazon Redshift cluster will be launched:

aws redshift restore-from-cluster-snapshot
  --region us-east-1
  --cluster-identifier cc-redshift-cluster-v2
  --snapshot-identifier cc-redshift-cluster-snapshot
  --node-type dc2.large
  --cluster-parameter-group-name default.redshift-1.0
  --cluster-subnet-group-name default
  --vpc-security-group-ids sg-abcdabcd
  --availability-zone us-east-1e
  --publicly-accessible

06 The command output should return the metadata available for the new Redshift cluster:

{
	"Cluster": {
		"ClusterIdentifier": "cc-redshift-cluster-v2",
		"NodeType": "dc2.large",
		"ClusterStatus": "creating",
		"ClusterAvailabilityStatus": "Modifying",
		"MasterUsername": "dbauser",
		"DBName": "redshiftdb",
		"AutomatedSnapshotRetentionPeriod": 1,
		"ManualSnapshotRetentionPeriod": -1,
		"ClusterSecurityGroups": [],
		"VpcSecurityGroups": [
			{
				"VpcSecurityGroupId": "sg-abcdabcd",
				"Status": "active"
			}
		],
		"ClusterParameterGroups": [
			{
				"ParameterGroupName": "default.redshift-1.0",
				"ParameterApplyStatus": "in-sync"
			}
		],
		"ClusterSubnetGroupName": "default",
		"VpcId": "vpc-12341234",
		"AvailabilityZone": "us-east-1e",
		"PreferredMaintenanceWindow": "wed:08:00-wed:08:30",
		"PendingModifiedValues": {},
		"ClusterVersion": "1.0",
		"AllowVersionUpgrade": true,
		"NumberOfNodes": 1,
		"PubliclyAccessible": true,
		"Encrypted": false,
		"Tags": [],
		"EnhancedVpcRouting": false,
		"IamRoles": [],
		"MaintenanceTrackName": "current",
		"DeferredMaintenanceWindows": [],
		"NextMaintenanceWindowStartTime": "2021-11-24T08:00:00+00:00",
		"AquaConfiguration": {
			"AquaStatus": "disabled",
			"AquaConfigurationStatus": "auto"
		}
	}
}

07 Run describe-clusters command (OSX/Linux/UNIX) with custom query filters to describe the database endpoint for the new Redshift database cluster:

aws redshift describe-clusters
  --region us-east-1
  --cluster-identifier cc-redshift-cluster-v2
  --query 'Clusters[*].Endpoint.Address'

08 The command output should return the requested database endpoint URL:

[
	"cc-redshift-cluster-v2.1234abcd1234.us-east-1.redshift.amazonaws.com"
]

09 As soon as the new Redshift cluster is provisioned, update your application configuration settings to point to the new cluster endpoint.

10 (Optional) To remove the EC2-Classic Redshift cluster (source cluster) from your AWS cloud account in order to eliminate any unnecessary expenses, run delete-cluster command (OSX/Linux/UNIX) using the name of the Redshift cluster that you want to delete as the identifier parameter:

aws redshift delete-cluster
  --region us-east-1
  --cluster-identifier cc-redshift-cluster
  --final-cluster-snapshot-identifier cc-redshift-cluster-final-snapshot

11 The command output should return the metadata available for the Redshift cluster selected for deletion:

{
	"Cluster": {
		"ClusterIdentifier": "cc-redshift-cluster",
		"NodeType": "dc2.large",
		"ClusterStatus": "final-snapshot",
		"ClusterAvailabilityStatus": "Modifying",
		"MasterUsername": "dbauser",
		"DBName": "redshiftdb",
		"Endpoint": {
			"Address": "cc-redshift-cluster-v2.abcd1234abcd.us-east-1.redshift.amazonaws.com",
			"Port": 5439
		},
		"AutomatedSnapshotRetentionPeriod": 1,
		"ManualSnapshotRetentionPeriod": -1,
		"PreferredMaintenanceWindow": "wed:08:00-wed:08:30",
		"PendingModifiedValues": {},
		"ClusterVersion": "1.0",
		"NumberOfNodes": 1,
		"PubliclyAccessible": true,
		"Encrypted": false,
		"Tags": [],
		"IamRoles": [],
		"MaintenanceTrackName": "current"
	}
}

12 Repeat steps no. 1 – 11 for each Redshift cluster that you want to re-create, provisioned in the selected AWS region.

13 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 12 to perform the Remediation process for other regions.

References

AWS Command Line Interface (CLI) Documentation
redshift
describe-clusters
create-cluster
delete-cluster

Publication date Oct 10, 2016

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS CloudFormation

Using Terraform (AWS Provider)

Using AWS Console

Using AWS CLI

References

Related Redshift rules