Kubernetes Cluster Logging

Risk Level: Low (generally tolerable level of risk)

Rule ID: EKS-003

Ensure that your Amazon Elastic Kubernetes Service (EKS) clusters have control plane logs enabled in order to publish API, audit, controller manager, scheduler or authenticator logs to AWS CloudWatch Logs. Amazon EKS control plane logging feature supports the following log types (each log type corresponds to a component within the Kubernetes control plane):

API server logs – these logs refer to the API requests made to your Amazon EKS cluster.

Audit logs – Kubernetes audit logs provide a record of the individual users, administrators, or system components that have interacted with your cluster via the Kubernetes API.

Authenticator logs – authenticator logs are unique to AWS EKS service. These logs to refer to authentication requests performed to the EKS cluster.

Controller manager logs – these logs contain information about the controller manager that handles the core control loopsthat are shipped with Kubernetes.

Scheduler logs – scheduler logs record when and where Kubernetes pods are running within your cluster.

This rule can help you with the following compliance standards:

PCI
HIPAA
GDPR
APRA

For further details on compliance standards supported by Conformity, see here.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Security

Once EKS Control Plane Logging feature is enabled, Amazon EKS sends audit and diagnostic logs directly to AWS CloudWatch Logs. These logs can help you to secure and efficiently run your EKS clusters. You can select the exact log types you need, and the logging data is sent as log streams to the AWS CloudWatch log group created for the specified Amazon EKS cluster.

Audit

To determine if control plane logging is enabled for your AWS EKS clusters in order to publish API, audit, controller manager, scheduler or authenticator logs to Amazon CloudWatch, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Amazon EKS dashboard at https://console.aws.amazon.com/eks/.

03 In the left navigation panel, under Amazon EKS, select Clusters.

04 Click on the name (link) of the EKS cluster that you want to examine to access the cluster configuration settings.

05 On the selected EKS cluster configuration page, within the Logging section, check the status for each control plane log type available (i.e. API server, Audit, Authenticator, Controller manager and Scheduler). If all the log types have their status set to Disabled, the EKS control plane logging is not enabled for the selected Amazon Elastic Kubernetes Service (EKS) cluster.

06 Repeat step no. 4 and 5 to determine the EKS control plane logging configuration status for other AWS EKS clusters available in the current region.

07 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Run list-clusters command (OSX/Linux/UNIX) using custom query filters to list the names of all AWS EKS clusters available in the selected region:

aws eks list-clusters
	--region us-east-1
	--output table
	--query 'clusters'

02 The command output should return a table with the requested EKS cluster identifiers:

-------------------------
|     ListClusters      |
+-----------------------+
|  cc-eks-prod-cluster  |
|  cc-eks-mobile-stack  |
+-----------------------+

03 Run describe-cluster command (OSX/Linux/UNIX) using the name of the EKS cluster that you want to examine as identifier parameter and custom query filters to obtain the control plane logging configuration status for the selected Amazon EKS resource:

aws eks describe-cluster
	--region us-east-1
	--name cc-eks-prod-cluster
	--query 'cluster.logging.clusterLogging[*].enabled'

04 The command output should return the feature status for the specified EKS cluster:

[
    false
]

If the describe-cluster command output returns only false, as shown in the output example above, the EKS service does not send API, audit, controller manager, scheduler or authenticator logs from the specified cluster to AWS CloudWatch Logs, therefore the EKS control plane logging is not enabled for the selected Amazon EKS cluster.

05 Repeat step no. 3 and 4 to check the EKS control plane logging configuration for other Amazon Elastic Kubernetes Service clusters available in the selected region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the audit process for other regions.

Remediation / Resolution

To enable EKS control plane logging for your Amazon Elastic Kubernetes Service (EKS) clusters, perform the following instructions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Amazon EKS dashboard at https://console.aws.amazon.com/eks/.

03 In the left navigation panel, under Amazon EKS, select Clusters.

04 Click on the name (link) of the EKS cluster that you want to reconfigure (see Audit section part I to identify the right EKS resource).

05 On the selected EKS cluster configuration page, click the Update button available in the Logging section to edit the EKS control plane logging configuration.

06 On the Update logging page, for each individual log type, choose whether the log type should be Enabled or Disabled. Cloud Conformity strongly recommends that you enable all the existing log types (i.e. API, audit, controller manager, scheduler and authenticator) when updating the EKS control plane logging feature configuration. Click UPDATE to apply the changes.

07 Repeat steps no. 4 – 6 to enable AWS EKS control plane logging for other Amazon EKS clusters available in the current region.

08 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Run update-cluster-config command (OSX/Linux/UNIX) to enable AWS EKS control plane logging feature for the selected Amazon EKS cluster (see Audit section part II to identify the right resource). Cloud Conformity strongly recommends that you include all the existing log types (i.e. API, audit, controller manager, scheduler and authenticator) when enabling the feature. The following update-cluster-config command example enables sending all available log types to AWS CloudWatch Logs service:

aws eks update-cluster-config
	--region us-east-1
	--name cc-eks-prod-cluster
	--logging '{"clusterLogging":[{"types":["api","audit","authenticator","controllerManager","scheduler"],"enabled":true}]}'

02 The command output should return the new configuration metadata available for the EKS control plane logging feature:

{
    "update": {
        "status": "InProgress",
        "errors": [],
        "params": [
            {
                "type": "ClusterLogging",
                "value": "{\"clusterLogging\":[{\"types\":[\"api\",\"audit\",\"authenticator\",\"controllerManager\",\"scheduler\"],\"enabled\":true}]}"
            }
        ],
        "type": "LoggingUpdate",
        "id": "abcd1234-abcd-1234-abcd-1234abcd1234",
        "createdAt": 1567539114.781
    }
}

03 Run describe-update command (OSX/Linux/UNIX) using the cluster name and the update ID returned at the previous step as identifier parameters to get the status of the log configuration update. The EKS control plane logging update is complete when the status is set to "Successful":

aws eks describe-update
	--region us-east-1
	--name cc-eks-prod-cluster
	--update-id abcd1234-abcd-1234-abcd-1234abcd1234
	--query 'update.status'

04 The command output should return the requested update status:

"Successful"

05 Repeat steps no. 1 – 4 to enable AWS EKS control plane logging for other Amazon EKS clusters provisioned in the selected region.

06 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.

References

AWS Command Line Interface (CLI) Documentation
eks
list-clusters
describe-cluster
update-cluster-config
describe-update

Publication date Sep 11, 2019

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS Console

Using AWS CLI

References

Related EKS rules