Use GKE Clusters with Private Endpoints Only

Risk Level: Medium (should be achieved)

Restrict control plane access to your Google Kubernetes Engine (GKE) clusters to private endpoints only, effectively disabling external access to the Kubernetes API. This requires configuring the GKE cluster with private nodes, a private master IP range, and IP aliasing.

Security

While Kubernetes API requires authentication tokens for sensitive actions, a vulnerability could expose it with unrestricted access. Moreover, attackers could potentially identify the cluster and Kubernetes API version to check for known vulnerabilities. Disabling the public endpoint, unless explicitly required, mitigates these risks by requiring attackers to be within the master's VPC network to target the Kubernetes API.

Audit

To determine if your Google Kubernetes Engine (GKE) clusters are using private endpoints only for control plane access, perform the following operations:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to examine from the console top navigation bar.

03 Navigate to Kubernetes Engine console available at https://console.cloud.google.com/kubernetes.

04 In the left navigation panel, under Resource Management, choose Clusters and select the OVERVIEW tab to access the list of GKE clusters provisioned for the selected GCP project.

05 Click on the name (link) of the GKE cluster that you want to examine.

06 Select the DETAILS tab to view the configuration information available for the selected cluster.

07 In the Control Plane Networking section, check the Public endpoint and Private endpoint attributes to determine the control plane network access configuration available for the selected cluster. If the Public endpoint attribute is set to a public IP address, the cluster control plane is also accessible from external IP addresses. Therefore, the selected Google Kubernetes Engine (GKE) cluster is not configured to use private endpoints only for control plane access.

08 Repeat steps no. 5 – 7 for each GKE cluster provisioned within the selected GCP project.

09 Repeat steps no. 2 – 8 for each GCP project deployed in your Google Cloud account.

Using GCP CLI

01 Run projects list command (Windows/macOS/Linux) with custom output filters to list the ID of each GCP project available in your Google Cloud account:

gcloud projects list
	--format="table(projectId)"

02 The command output should return the requested GCP project IDS:

PROJECT_ID
cc-web-project-123123
cc-dev-project-112233

03 Run container clusters list command (Windows/macOS/Linux) with the ID of the GCP project that you want to examine as the identifier parameter and custom output filters to describe the name and the region of each GKE cluster provisioned for the selected project:

gcloud container clusters list
	--project cc-web-project-123123
	--format="table(NAME,ZONE)"

04 The command output should return the requested cluster names and their regions:

NAME: cc-gke-backend-cluster
ZONE: us-central1

NAME: cc-gke-frontend-cluster
ZONE: us-central1

05 Run container clusters describe command (Windows/macOS/Linux) with the name of the GKE cluster that you want to examine as the identifier parameter and custom output filters to determine the control plane network access configuration available for the selected cluster:

gcloud container clusters describe cc-gke-backend-cluster
	--region=us-central1
	--format="json(endpoint,privateClusterConfig.enablePrivateEndpoint)

06 The command output should return the IP address of the control plane endpoint and the configuration status of the private endpoint. A status of "enablePrivateEndpoint": true indicates that the private endpoint is enabled and public access is disabled:

{
	"endpoint": "34.60.107.170"
}

If the container clusters describe command output does not return a private IP address for "endpoint", and the private endpoint status is not set to true ("enablePrivateEndpoint": true), the cluster is not configured with private endpoints only. In this case, the command output will return a public IP address for "endpoint". This indicates that the selected Google Kubernetes Engine (GKE) cluster is not configured to use private endpoints only for control plane access.

07 Repeat steps no. 5 and 6 for each GKE cluster provisioned for the selected GCP project.

08 Repeat steps no. 3 – 7 for each GCP project deployed in your Google Cloud account.

Remediation / Resolution

To ensure that your Google Kubernetes Engine (GKE) clusters are using private endpoints only for control plane access, you must re-create your clusters with the appropriate networking configuration, as described below:

Using GCP Console

01 Sign in to the Google Cloud Management Console.

02 Select the Google Cloud Platform (GCP) project that you want to access from the console top navigation bar.

03 Navigate to Kubernetes Engine console available at https://console.cloud.google.com/kubernetes.

04 In the left navigation panel, under Resource Management, choose Clusters and select the OVERVIEW tab to access the list of GKE clusters deployed for the selected GCP project.

05 Click on the name (link) of the GKE cluster that you want to re-create.

06 Select DUPLICATE from the console top menu to initiate the setup process using the same configuration settings as the source cluster and perform the following changes:

For Cluster basics, provide a unique name for the new GKE cluster in the Name box.
For Networking, under Control Plane Access, perform the following actions to define from where you can access the control plane:
1. Check the Access using DNS setting checkbox to enable the DNS-based control plane access.
2. Check the Access using IPv4 addresses setting checkbox, choose SHOW IP ADDRESS FIREWALL RULES, and configure the following options:
  1. Check the Access using the control plane's internal IP address from any region checkbox to provide access to the control plane's internal endpoint from any GCP region in a VPC or on-premises environment.
  2. Check the Enforce authorized networks on the control plane's internal endpoint setting checkbox and choose ADD AN AUTHORIZED NETWORK under Authorized networks to configure authorized networks in order to restrict access to the cluster's private control plane endpoint.
3. Under Cluster networking, check the Enable Private nodes setting checkbox to provision all cluster nodes with only internal IP addresses. This feature prevents external clients from accessing the nodes and from the nodes having direct access to the Internet.
Choose CREATE from the console bottom menu to create your new Google Kubernetes Engine (GKE) cluster.

07 (Optional) Once the new GKE cluster is operating successfully, you can remove the source cluster in order to stop adding charges to your Google Cloud bill:

Navigate back to the Clusters page and select the OVERVIEW tab to list the GKE clusters deployed for the selected GCP project.
Select the cluster that you want to delete (i.e., source cluster), and choose DELETE from the console top menu.
In the confirmation box, confirm deletion by typing the cluster name in the required text box, and choose DELETE to remove the selected GKE cluster.

08 Repeat steps no. 5 – 7 for each GKE cluster that you want to re-create, created for the selected GCP project.

09 Repeat steps no. 2 – 8 for each GCP project available in your Google Cloud account.

Using GCP CLI

01 Run container clusters describe command (Windows/macOS/Linux) with the name of the GKE cluster that you want to re-create as the identifier parameter, to describe the configuration information available for the selected cluster:

gcloud container clusters describe cc-gke-backend-cluster
	--region=us-central1
	--format="json"

02 The command output should return the requested configuration information:

{
	"addonsConfig": {
		"gcePersistentDiskCsiDriverConfig": {
			"enabled": true
		},
		"kubernetesDashboard": {
			"disabled": true
		},
		"networkPolicyConfig": {
			"disabled": true
		}
	},
	"autopilot": {},
	"autoscaling": {
		"autoscalingProfile": "BALANCED"
	},

	...

	"releaseChannel": {
		"channel": "REGULAR"
	},
	"securityPostureConfig": {
		"mode": "DISABLED",
		"vulnerabilityMode": "VULNERABILITY_MODE_UNSPECIFIED"
	},
	"selfLink": "https://container.googleapis.com/v1/projects/cc-web-project-123123/locations/us-central1/clusters/cc-gke-backend-cluster",
	"shieldedNodes": {
		"enabled": false
	},
	"status": "RUNNING",
	"zone": "us-central1"
}

03 Run container clusters create command (Windows/macOS/Linux) with the information returned at the previous step as the configuration data for the command parameters, to create a new Google Kubernetes Engine (GKE) cluster that is using private endpoints only for control plane access. Include the --enable-master-authorized-networks and --master-authorized-networks command parameters to specify the authorized networks used to restrict access to the cluster's private control plane endpoint:

gcloud container clusters create cc-private-gke-backend-cluster
	--project=cc-web-project-123123
	--region=us-central1
	--disk-size=150
	--machine-type=e2-micro
	--num-nodes=3
	--release-channel=regular
	--enable-dns-access
	--enable-ip-access
	--enable-master-global-access
	--enable-master-authorized-networks
	--master-authorized-networks 198.51.100.0/24
	--enable-ip-alias
	--enable-private-nodes
	--enable-private-endpoint

04 The command output should return the full URL of the new GKE cluster:

Created [https://container.googleapis.com/v1/projects/cc-web-project-123123/zones/us-central1/clusters/cc-private-gke-backend-cluster]

05 (Optional) Once the new GKE cluster is operating successfully, you can run container clusters delete command (Windows/macOS/Linux) to remove the source cluster in order to stop adding charges to your Google Cloud bill:

gcloud container clusters delete cc-gke-backend-cluster
	--region=us-central1

06 Type Y (Yes) to confirm the removal:

The following clusters will be deleted.
- [cc-gke-backend-cluster] in [us-central1]

Do you want to continue (Y/n)?  Y

07 The command output should return the URL of the deleted GKE cluster:

Deleted [https://container.googleapis.com/v1/projects/cc-web-project-123123/zones/us-central1/clusters/cc-gke-backend-cluster]

08 Repeat steps no. 1 - 4 for each GKE cluster that you want to re-create, available within the selected GCP project.

09 Repeat steps no. 1 – 8 for each GCP project deployed in your Google Cloud account.

References

Google Cloud Platform (GCP) Documentation
Harden your cluster's security
Customize your network isolation in GKE

GCP Command Line Interface (CLI) Documentation
gcloud projects list
gcloud container clusters list
gcloud container clusters describe
gcloud container clusters create
gcloud container clusters delete

Publication date Jan 6, 2025

Audit

Using GCP Console

Using GCP CLI

Remediation / Resolution

Using GCP Console

Using GCP CLI

References

Related GKE rules