Enable Support for Network Policies

Risk Level: Medium (should be achieved)

Rule ID: AlibabaCloud-ACK-007

Ensure that your ACK clusters are using Kubernetes network policies to implement secure policy-based access control. Container Service for Kubernetes (ACK) employs the Terway network plugin to enforce network policies at the cluster level.

Security

A network policy is a Kubernetes resource that allows you to control the traffic between pods and services within a cluster. Network policies are useful for enforcing security policies, isolating applications, and debugging network connectivity issues. By default, pods are not isolated, meaning they accept traffic from any source. Isolation is achieved by applying a network policy to select pods. Once a network policy is in place for a namespace, specifying a particular pod, that pod will reject any connections not permitted by the applied network policy.

Audit

To determine if your ACK clusters are using network policies, perform the following operations:

Using Alibaba Cloud Console

01 Sign in to your Alibaba Cloud account.

02 Navigate to Container Service for Kubernetes (ACK) console at https://cs.console.aliyun.com.

03 In the left navigation panel, under Overview, choose Clusters.

04 Click on the name (link) of the ACK cluster that you want to examine, listed in the Cluster Name/ID column.

05 In the ACK resource navigation panel, under the cluster name, choose Cluster information.

06 Choose the Basic Information tab and check the Network Plug-in attribute value. If the Network Plug-in value is not Terway, the cluster is not using the Terway network plugin to enforce network policies, therefore the selected cluster is not using network policies. If Network Plug-in is set to Terway, choose ConfigMap from the left navigation panel under Configurations, select the kube-system namespace, click on eni-config ConfigMap, and check the disable_network_policy parameter value. If the disable_network_policy value is set to true, the selected ACK cluster is not configured to use network policies.

07 Repeat steps no. 4 – 6 for each Container Service for Kubernetes (ACK) cluster available within your Alibaba Cloud account.

Using Alibaba Cloud CLI

01 Run GET /clusters command (OSX/Linux/UNIX) to describe the configuration details for each Container Service for Kubernetes (ACK) cluster provisioned in your Alibaba Cloud account:

aliyun cs GET /clusters
  --header "Content-Type=application/json;"
  --body "{}"

02 The command output should return the configuration information available for each available ACK cluster:

[
	{
		"cluster_id": "abcd1234abcd1234abcd1234abcd1234a",
		"cluster_spec": "ack.standard",
		"cluster_type": "ManagedKubernetes",
		"created": "2024-02-05T17:44:26+08:00",
		"current_version": "1.28.3-aliyun.1",
		"deletion_protection": false,
		"init_version": "1.28.3-aliyun.1",
		"parameters": {
		"ALIYUN::AccountId": "1234567890123456",
		"ALIYUN::NoValue": "None",
		"ALIYUN::Region": "eu-west-1",
		"ALIYUN::TenantId": "1234567890123456",
		"AdjustmentType": "TotalCapacity",
		"BetaVersion": "",
		"CloudMonitorFlags": "False",
		"CloudMonitorVersion": "1.3.7",
		"ClusterDns": "192.168.0.10",
		"ClusterId": "abcd1234abcd1234abcd1234abcd1234a",
		"ContainerCIDR": "",
		"CustomK8sWorkerRole": "",
		"DisableAddons": "True",
		"DisableAutoCreateK8sWorkerRole": "False",
		"DisableAutoCreateK8sWorkerRolePolicy": "True",
		"DockerVersion": "17.06.2-ce-3",
		"ESSDeletionProtection": "True",
		"Eip": "False",
		"EipAddress": "",
		"EtcdVersion": "v3.5.9",
		"ExecuteVersion": "922993032",
		"HealthCheckType": "NONE",
		"IPStack": "ipv4",
		"ImageId": "aliyun_3_9_x64_20G_alibase_20231219.vhd",
		"KubernetesVersion": "1.28.3-aliyun.1",
		"MasterSLBPrivateIP": "172.23.38.234",
		"NatGateway": "False",
		"NatGatewayId": "",
		"NatGatewayType": "Enhanced",
		"NatGatewayVswitchId": "",
		"Network": "Flannel",
		"NodeNameMode": "nodeip",
		"NumOfNodes": "0",
		"OSType": "Linux",
		"Password": "******",
		"ProtectedInstances": "",
		"ProxyMode": "ipvs",
		"RemoveInstanceIds": "",
		"SNatEntry": "False",
		"ServiceCIDR": "192.168.0.0/16",
		"UserData": "",
		"VpcCidrWithSecondaryCidrs": "[\"172.16.0.0/12\"]",
		"WorkerAutoRenew": "False",
		"WorkerAutoRenewPeriod": "1",
		"WorkerDataDisk": "False",
		"WorkerDataDisks": "[]",
		"WorkerDeletionProtection": "True",
		"WorkerDeploymentSetId": "",
		"WorkerHpcClusterId": "",
		"WorkerInstanceChargeType": "PostPaid",
		"WorkerInstanceTypes": "ecs.ic5.xlarge",
		"WorkerKeyPair": "",
		"WorkerLoginPassword": "******",
		"WorkerPeriod": "3",
		"WorkerPeriodUnit": "Month",
		"WorkerSnapshotPolicyId": "******",
		"WorkerSystemDiskCategory": "cloud_essd",
		"WorkerSystemDiskPerformanceLevel": "PL0",
		"WorkerSystemDiskSize": "20",
		"ZoneId": ""
		},
		"profile": "Default",
		"region_id": "eu-west-1",
		"size": 1,
		"state": "running",
		"updated": "2024-02-05T17:46:49+08:00",
		"zone_id": "eu-west-1a"
	},

	...

	{
		"cluster_id": "1234abcd1234abcd1234abcd1234abcd1",
		"cluster_spec": "ack.standard",
		"cluster_type": "ManagedKubernetes",
		"created": "2024-02-05T16:40:31+08:00",
		"current_version": "1.28.3-aliyun.1",
		"deletion_protection": false,
		"init_version": "1.28.3-aliyun.1",
		"parameters": {
		"ALIYUN::AccountId": "1234567890123456",
		"ALIYUN::NoValue": "None",
		"ALIYUN::Region": "eu-west-1",
		"ALIYUN::StackName": "k8s-for-cs-1234abcd1234abcd1234abcd1234abcd1",
		"ALIYUN::TenantId": "1234567890123456",
		"AdjustmentType": "TotalCapacity",
		"BetaVersion": "",
		"CloudMonitorFlags": "False",
		"CloudMonitorVersion": "1.3.7",
		"ClusterDns": "10.0.0.10",
		"ClusterId": "1234abcd1234abcd1234abcd1234abcd1",
		"ContainerCIDR": "10.65.0.0/16",
		"CustomK8sWorkerRole": "",
		"DisableAddons": "True",
		"DisableAutoCreateK8sWorkerRole": "False",
		"DisableAutoCreateK8sWorkerRolePolicy": "True",
		"DockerVersion": "17.06.2-ce-3",
		"ESSDeletionProtection": "True",
		"Eip": "False",
		"EipAddress": "",
		"EtcdVersion": "v3.5.9",
		"ExecuteVersion": "460378767",
		"HealthCheckType": "NONE",
		"IPStack": "ipv4",
		"KeyPair": "",
		"KubernetesVersion": "1.28.3-aliyun.1",
		"MasterSLBPrivateIP": "172.23.38.230",
		"NatGateway": "False",
		"NatGatewayId": "",
		"NatGatewayType": "Enhanced",
		"NatGatewayVswitchId": "",
		"Network": "Flannel",
		"NodeNameMode": "nodeip",
		"NumOfNodes": "0",
		"OSType": "Linux",
		"Password": "******",
		"PodVswitchIds": "[]",
		"ProtectedInstances": "",
		"ProxyMode": "ipvs",
		"RemoveInstanceIds": "",
		"SNatEntry": "False",
		"ServiceCIDR": "10.0.0.0/16",
		"SnatTableId": "",
		"UserData": "",
		"VpcCidrWithSecondaryCidrs": "[\"172.16.0.0/12\"]",
		"WorkerAutoRenew": "False",
		"WorkerAutoRenewPeriod": "1",
		"WorkerDataDisk": "False",
		"WorkerDataDisks": "[]",
		"WorkerDeletionProtection": "True",
		"WorkerDeploymentSetId": "",
		"WorkerHpcClusterId": "",
		"WorkerInstanceChargeType": "PostPaid",
		"WorkerInstanceTypes": "ecs.u1-c1m1.xlarge",
		"WorkerKeyPair": "",
		"WorkerLoginPassword": "******",
		"WorkerPeriod": "3",
		"WorkerPeriodUnit": "Month",
		"WorkerSnapshotPolicyId": "******",
		"WorkerSystemDiskCategory": "cloud_essd",
		"WorkerSystemDiskPerformanceLevel": "PL0",
		"WorkerSystemDiskSize": "120",
		"ZoneId": ""
		},
		"profile": "Default",
		"region_id": "eu-west-1",
		"size": 1,
		"state": "running",
		"subnet_cidr": "10.65.0.0/16",
		"updated": "2024-02-05T16:42:53+08:00",
		"zone_id": "eu-west-1a"
	}
]

Check the network plugin configured for each ACK cluster by examining the "Network" attribute listed within the cluster parameters (i.e. "parameters" object). If the "Network" attribute value is not "terway-eniip", the cluster is not using the Terway network plugin to enforce network policies, therefore the selected cluster is not using network policies. If the "Network" attribute value is set to "terway-eniip", the Terway network plugin is installed on your cluster, therefore you can continue the Audit process with the next step.

03 Run GET /clusters/[cluster_id] command (OSX/Linux/UNIX) with the ID of the ACK cluster that you want to examine as the identifier parameter, to describe the configuration metadata available for the selected cluster:

aliyun cs GET /clusters/abcd1234abcd1234abcd1234abcd1234a
  --header "Content-Type=application/json;"
  --body "{}"
  --output cols=meta_data

04 The command output should return the configuration information available for each cluster node pool:

meta_data
---------
{
	"Addons": [
		{
			"name": "nginx-ingress-controller",
			"disabled": true
		},
		{
			"name": "terway-eniip",
			"version": "v1.6.3",
			"config": "{\"ENITrunking\":\"false\",\"IPVlan\":\"false\",\"NetworkPolicy\":\"false\"}"
		},
		{
			"name": "managed-kube-proxy",
			"version": "v1.28.3-aliyun.1"
		},
		{
			"name": "csi-plugin",
			"version": "v1.28.3-eb95171-aliyun"
		},
		{
			"name": "gateway-api",
			"version": "1.0.1"
		},
		{
			"name": "cloud-controller-manager",
			"version": "v2.8.1-mgk"
		},
		{
			"name": "ack-helm-manager",
			"version": "0.1.2-d12e6ac2"
		},
		{
			"name": "ack-scheduler",
			"version": "v1.28.3-aliyun-6.3.5b0df817"
		}
	],
	"AuditProjectName": "",
	"DockerVersion": "",
	"EtcdVersion": "v3.5.9",
	"ExtraCertSAN": null,
	"HasSandboxRuntime": false,
	"IPStack": "ipv4",
	"ImageType": "AliyunLinux3",
	"KubernetesVersion": "1.28.3-aliyun.1",
	"MultiAZ": false,
	"NameMode": "",
	"NextVersion": "",
	"OSType": "Linux",
	"Platform": "AliyunLinux",
	"Provider": "",
	"RRSAConfig": {
		"enabled": false
	},
	"Runtime": "containerd",
	"RuntimeVersion": "1.6.28",
	"ServiceCIDR": "192.168.0.0/16",
	"SubClass": "default",
	"SupportPlatforms": [
		"CentOS",
		"AliyunLinux",
		"Windows",
		"WindowsCore"
	],
	"VpcCidr": "172.16.0.0/12"
}

Check the "config" configuration object returned for the Terway network plugin/addon, e.g. {"name": "terway-eniip","version": "v1.6.3","config": "{\"ENITrunking\":\"false\",\"IPVlan\":\"false\",\"NetworkPolicy\":\"false\"}"}. If the "NetworkPolicy" attribute value is set to "false", the selected ACK cluster is not configured to use network policies.

Remediation / Resolution

To ensure that ACK clusters are using network policies, perform the following operations:

Using Alibaba Cloud Console

01 Sign in to your Alibaba Cloud account.

02 Navigate to Container Service for Kubernetes (ACK) console at https://cs.console.aliyun.com.

03 In the left navigation panel, under Overview, choose Clusters.

04 If the Terway network plugin is installed on your ACK clusters, perform the following actions:

Click on the name (link) of the ACK cluster that you want to configure, listed in the Cluster Name/ID column.
In the ACK resource navigation panel, under Configurations, choose ConfigMaps.
Select the kube-system namespace from the top menu, choose Edit next to eni-config, set disable_network_policy to false to enable support for network policies, and choose OK to apply the changes.

05 If the Terway network plugin is not installed on your ACK clusters, you must re-create your clusters with Terway by performing the following actions:

Choose Create Kubernetes Cluster and follow the setup wizard to create a new ACK cluster.
On the Cluster Configurations setup page, choose Terway for Network Plug-in and select the Support for NetworkPolicy setting checkbox to enable support for network policies.
After all the required settings are configured, choose Create Cluster to deploy your new ACK cluster.

06 Repeat steps no. 4 and 5 for each Container Service for Kubernetes (ACK) cluster available in your Alibaba Cloud account.

Using Alibaba Cloud CLI

01 Run POST /clusters command (OSX/Linux/UNIX) to create a new ACK cluster with support for network policies. Include the following parameters in the command request: "addons":[{"name":"terway-eniip","config":"{"NetworkPolicy":"true"}"}] and "pod_vswitch_ids":["vsw-1234abcd1234abcd1234a"] to install the Terway network plugin on the new cluster. "pod_vswitch_ids" parameter is essential when Terway is installed on the cluster, as it ensures that each pod within the cluster is assigned a distinct IP address:

aliyun cs POST /clusters
  --header "Content-Type=application/json;"
  --body "{\"name\":\"tm-project-new-cluster\",\"region_id\":\"eu-west-1\",\"cluster_type\":\"Kubernetes\",\"vpcid\":\"vpc-1234abcd1234abcd1234a\",\"service_cidr\":\"192.168.0.0/16\",\"cluster_spec\":\"ack.standard\",\"kubernetes_version\":\"1.28.3-aliyun.1\",\"vswitch_ids\":[\"vsw-1234abcd1234abcd1234a\"],\"nodepools\":[{\"auto_scaling\":{\"enable\":false,\"max_instances\":3,\"min_instances\":1,\"type\":\"cpu\"},\"kubernetes_config\":{\"cms_enabled\":true,\"runtime\":\"containerd\",\"runtime_version\":\"1.6.28\"},\"nodepool_info\":{\"name\":\"tm-default-node\"},\"scaling_group\":{\"instance_charge_type\":\"PostPaid\",\"instance_types\":[\"ecs.n1.medium\"],\"vswitch_ids\":[\"vsw-1234abcd1234abcd1234a\"]}}],\"addons\":[{\"name\":\"terway-eniip\",\"config\":\"{\\\"NetworkPolicy\\\":\\\"true\\\"}\"}],\"pod_vswitch_ids\":[\"vsw-1234abcd1234abcd1234a\"]}"

02 If successful, the output should return the new ACK cluster ID:

{
	"cluster_id": "1234abcd1234abcd1234abcd1234abcd1",
	"request_id": "ABCDABCD-1234-ABCD-1234-ABCD1234ABCD",
	"task_id": "T-abcdabcdabcdabcdabcdabcd"
}

03 Repeat steps no. 1 and 2 for each ACK cluster that you want to re-create, available within your Alibaba Cloud account.

References

Alibaba Cloud CLI Documentation
DescribeClusterDetail
CreateCluster

Publication date Feb 22, 2024

Audit

Using Alibaba Cloud Console

Using Alibaba Cloud CLI

Remediation / Resolution

Using Alibaba Cloud Console

Using Alibaba Cloud CLI

References

Related AlibabaCloud-ACK rules