Protecting Your Krew: A Security Analysis of kubectl Plug-ins

By Magno Logan

In this article, we analyze the security of kubectl plug-ins and their plug-in manager called Krew. First, we briefly discuss kubectl and the Krew plug-in manager, how they work, and their primary use. Afterward, we list Krew plug-ins for which particular care is necessary during their use. We also show some risks that we found based on source code and software composition analysis. Lastly, we show some possible supply-chain attack scenarios related to Krew and kubectl plug-ins.

What is kubectl?

Anyone who has used Kubernetes might have also already used the command-line tool kubectl. The command line can be pronounced in many ways, the main ones being “kube-c-t-l,” “kube-control,” or “kube-cuddle.” Like the Docker command-line interface (CLI) that allows one to manage their Docker containers, kubectl also enables the management of Kubernetes clusters (aka K8s). It uses a file named “config,” which is in the $HOME/.kube folder by default. However, it is also possible to use other kubeconfig files via the environment variable or the --kubeconfig flag.

What is Krew?

Krew is the plug-in manager for kubectl CLI that helps discover kubectl plug-ins, installs them on a user’s machine, and keeps them up to date. It also helps plug-in developers package and distribute their plug-ins on many different platforms efficiently, making them visible via the Krew’s centralized repository. As of this writing, there are 154 plug-ins that are available and distributed on Krew.

The plug-ins have different goals and functionalities, but they all try to improve and speed up the Kubernetes experience and help manage K8s clusters more efficiently. However, anyone can develop their kubectl plug-in and share it via Krew. Notably, there are no security checks for these.

After installing a plug-in, a user simply receives a warning that reminds them to be careful when selecting which plug-ins to download and use, as these might contain malicious code or might perhaps even be compromised.

Figure 1. The only warning issued after installing a kubectl plug-in via Krew

List of Krew plug-ins

As of September, the following is a list of the top 10 most popular kubectl plug-ins available on Krew according to the number of stars on GitHub:

Table 1. Top 10 most popular kubectl plug-ins available on Krew based on GitHub stars

Although GitHub stars don’t measure usage, we can see that some of these plug-ins are popular based on the star rating on their repositories, which are in the thousands. Meanwhile, there are also less-known or brand-new plug-ins that have yet to gain popularity. Most of these plug-ins are created by both the community and individuals interested in Kubernetes and open source. Others are maintained by organizations, such as cert-manager, Flyte, and Kyverno.

As previously mentioned, it is interesting that anyone can develop their kubectl plug-in and share it via Krew, as this raises an important question: What if someone creates a malicious plug-in that does what it is expected to do but collects sensitive data, such as the ~/.kube/config files, from whoever installs it? Since there are no guarantees or security checks for the submitted plug-ins, this would be a significant supply-chain risk. While we have decided not to test the approach for ethical purposes, here is the threat model that we have discussed and created for this scenario:

Figure 2. Threat model scenario of sending a malicious plug-in to Krew

Source code risks

After analyzing these plug-ins’ source code vulnerabilities, it becomes clear that security gaps in the source code can bring similar risks to downloading and executing code from the internet without verifying its integrity and provenance. Since these are kubectl plug-ins, users install them in their local machines or wherever kubectl is running.

KICS results

We checked the code of the Krew plug-ins that Kubernetes users and developers can use for their projects with the help of some available tools online. The first one is Keep Infrastructure as Code Secure (KICS) by Checkmarx. KICS scans projects for security vulnerabilities, compliance issues, and infrastructure misconfigurations early in the development cycle of infrastructure as code (IaC).

After removing the obvious false positives, we found 153 issues, each of them affecting multiple files and multiple plug-ins out of the 154 Krew plug-ins that we scanned with KICS. However, it must be noted that KICS only reports the failed checks that it finds. We list the issues identified by severity.

Figure 3. KICS results of the Krew plug-ins by severity

Of all the issues identified by KICS, 45% were flagged as having medium severity, and over 22% had low severity in source code risks. Meanwhile, high-severity issues were at 16%. The severity values are defined by each rule and are hard-coded inside the tool. When we look at the tests divided by platform, we can see that most of them are related to Kubernetes, which makes sense since we are talking about Kubernetes plug-ins. Other issues are related to the API, Terraform, and Dockerfiles that are used by these plug-ins.

Figure 4. KICS results by platform from the scan of Krew plug-ins

It is clear here that some of the Krew plug-ins do not drop capabilities for their containers. However, upon closer examination, it becomes evident that Linux capabilities were created to allow for more granular access control to privileged processes. Before that, options for Linux were only allowed for root or non-root. Therefore, one best practice for users to avoid this is to run containers as root or privileged processes in the container world. Additionally, limiting the capabilities of containers can help minimize an incident’s impact and reduce the blast radius.

KICS also detected a security issue involving some plug-ins deploying pods or containers without a security context. A security context defines privilege and access control settings for a pod, a container, or a set of security configurations that can and should be applied to pods on a cluster. As with capabilities, lacking a security context on pods can increase the damage that attackers can inflict in case of compromise.

Finally, role-based access control (RBAC) or role-based security with permissions to read secrets can also be a significant red flag that needs attention when using these plug-ins. Kubernetes uses RBAC to control who can access what is on a cluster. Like web applications, the Kubernetes RBAC comprises Roles — a set of permissions and actions that are allowed to perform on the cluster — and RoleBindings that connect or bind the Roles to a group of users or service accounts. Roles are applied to each namespace in a K8s cluster. But two other objects that can be used to apply cluster-level permissions are ClusterRole and ClusteRoleBinding.

What about secrets? Secrets are Kubernetes objects that contain sensitive data such as a password, a token, or a key. They are only base64-encoded and are not encrypted by default on etcd, an open-source key-value store. This means that having the ability to read secrets should be restricted to only a few users and service accounts that are required to follow the principle of least privilege for them to perform their tasks.

Figure 5. Categories of issues detected by KICS from Krew plug-ins

When we look at the categories of the issues detected by KICS, we see that the top three main problems are related to “Insecure Configurations,” “Best Practices,” and “Supply Chain.” This comes as no surprise as misconfigurations are one of the main issues in cloud-native environments. According to OWASP’s Top 10: 2021 List, security misconfigurations remain in the top 10 risks to web applications.

Following best practices is always essential to achieve proper security, especially when relying on third-party plug-ins to help manage clusters. And since supply-chain attacks have increased and have been placed in the spotlight since the Sunburst attack on SolarWinds in 2020, companies and their partners must ensure that all aspects of their operations’ security are checked and validated, keeping in mind the integrity and provenance of plug-ins, dependencies, and containers’ images.

Trend Micro Cloud One™ – Open Source Security results

When discussing dependency security, we need to be aware that most of these projects rely on libraries and dependencies based on open-source software. These libraries are usually incorporated during the development life cycle and are rarely updated or checked against known vulnerabilities.  

In this section, we analyze the dependencies of the kubectl plug-ins and look for outdated or vulnerable libraries and license risks that they might be using in their code. We also look at issues in their Dockerfiles and YAML-based files. For this analysis, we used Trend Micro Cloud One™ – Open Source Security by Snyk to scan and get the security analysis results. Here are the results of the top 15 Krew plug-ins with the highest number of vulnerabilities:

Figure 6. Top 15 Krew plug-ins with the highest-severity vulnerabilities as scanned by Trend Micro Cloud One – Open Source Security by Snyk

To be clear, not all the issues are directly related to the Krew plug-ins. However, some of those in the results can be linked to the base image used in the Dockerfile of these plug-ins. For example, the Dockerfile of the kubectl-match-name plug-in uses an old version of the Golang Docker image. There are also other images with fewer packages that are significantly safer.

Figure 7. List of vulnerabilities on different Golang Docker images. Screenshot taken from Trend Micro Cloud One – Open Source Security by Snyk.

The latest version of the kubectl-match-name plug-in has a high-severity vulnerability in one of its libraries called “crypto.” Given a Common Vulnerability Scoring System (CVSS) of 8.6 out of 10, CVE-2020-9283 is identified as an Improper Signature Verification in Go, wherein a malicious actor can attack a Secure Shell (SSH) server that accepts public keys. A proof of concept is already published on Exploit DB, so companies using this plug-in for their projects should fix this as soon as possible.

Figure 8. CVE-2020-9283 is ranked as a high-severity vulnerability found in one of the libraries from the kubectl-match-name plug-in

Conclusion

Although most kubectl plug-ins are helpful and valuable, one can never be too careful about using them. Before downloading and using any plug-in from Krew, it is still advisable to check its code and the number of users using it. Likewise, it is essential to check if a plug-in has been updated lately. In our case, some of the plug-ins that we checked have not been updated for over a year, which might increase their chances of having vulnerabilities or, at the very least, outdated libraries. Therefore, it is essential to run the command kubectl krew upgrade, which updates the local copy of the plug-in index. The command then checks the versions of the plug-ins installed and updates them if necessary.

Figure 9. Running kubectl krew upgrade command to update Krew plug-ins

Trend Micro solutions

Developers are pulling in numerous open-source dependencies for the creation and maintenance of their projects and applications, as well as for the support of operations even without security controls or visibility. Trend Micro Cloud One – Open Source Security by Snyk reduces the ability of adversaries to infiltrate software while supporting developers to build, monitor, and manage more robust applications. It allows both security and DevOps teams to see all direct and indirect dependency paths by scanning projects in code repositories to identify vulnerabilities in the open-source code used by the organization and how to fix them. By gaining visibility, security operations (SecOps) teams eliminate blind spots and generate automated reports immediately to prioritize threats detected early in the pipeline and predeployment stages.