Exposed Container Registries: A Potential Vector for Supply-Chain Attacks

By Alfredo de Oliveira and David Fiser

In this entry, we will discuss publicly exposed registries, which are repositories or databases containing information accessible to the public without the need for authentication. We discovered a substantial number of these registries hosted on cloud services, from which we were able to successfully download the contents stored inside them, including over 9 TB worth of downloaded images and more than 20,000 dumped images.

This entry sheds light on the potential security implications and highlights the importance of understanding the security considerations when using public registries. We will not focus on the security of dedicated cloud-hosted container registries. Instead, we will examine the pitfalls of manual container registry deployments inside cloud workloads since they are inherently more prone to misconfigurations and pose a valid attack vector for threat actors.

Here are some key initial statistics:

Figure 1. The content downloaded from the publicly exposed registries

Containers have revolutionized the way we develop, deploy, and manage applications. By offering isolation, portability, and consistency, they allow for rapid scaling and efficient integration in cloud environments. Containers have driven the growth of cloud-native technologies, fundamentally changing how many modern applications are designed, deployed, and managed.

An important aspect of working with containers is the matter of storing and managing the container images used to encapsulate applications. Since these images have to be stored somewhere, cloud service providers (CSPs) have started offering managed registry services that act as image repositories and allow developers to share and deploy applications across various environments. In addition, companies can choose to deploy and operate their own container image registries, either on-premises or inside cloud computing instances workloads (such as Amazon EC2).

Understanding container image registries

Container image registries are more than just repositories; they are the lifeblood of any containerized system. These registries store container images, which are static files that contain layers with pre-installed applications together with their dependencies and environmental setups. The setups include application code, libraries, system tools, and settings.

Each layer of a container image represents a modification to the image, like the addition of a file or a change in configuration. These layers are stacked on top of each other to form the final image. When an image is updated, only the layers that have changed are updated, which makes the use of container images efficient.

The creation of a container typically begins with the building of an image for an application. This process involves compiling application code, bundling it with the necessary dependencies, and configuring the runtime environment. The resulting image is a standalone executable package that encapsulates everything needed to run an application.

Once the image is built, it is then pushed to a registry (a private or public repository). CSPs can offer either or both as a managed service, depending on the need and configuration. Note that users can choose how to configure access to the registries (even public repositories limit who can push changes or upload images) and are also responsible for filtering possible sensitive content before it is pushed into the repository.

The registry acts as a distribution point, storing the image and allowing it to be pulled when needed. This is a critical aspect of the container life cycle, as it ensures that different versions of applications can consistently and reliably be deployed across different environments.

Figure 2. How container image registries work

The image stored in the registry encapsulates the application and all its software dependencies. This means that the application will run the same way, regardless of the environment it is deployed in, which is made possible by using container image registries.

Container image registries play a pivotal role in the deployment of applications in a containerized environment and act as the centralized storage and distribution point for container images.

When a container is launched, the system reaches out to the registry in order to pull the required container image. This image is then used to instantiate a running container on the host system. Given this, the security and reliability of the registry are of utmost importance. If a registry is compromised, it can lead to the deployment of compromised images. This can have far-reaching security implications that we cover in the second part of our research.

However, the role of a registry extends beyond just storage. A registry also allows the storage of multiple image versions and the tracking of image variants using tags. For example, a developer might have different images for development, testing, and production environments, each with slightly different configurations. The registry allows the tracking of all these variants, thereby also allowing developers to pull the appropriate image as needed.

Figure 3. A container registry and its container layers

However, there are also security risks in using cloud-based services. For example, in container image registries, if a malicious actor gains either write access or the ability to push images, they can potentially inject malicious code into the image and replace the original image inside the registry. On the other hand, if they gain read access to a registry that should be private, they can pull the images and abuse sensitive information. This is why it's crucial to secure not just the registry itself but also the individual images stored in it.

Since container image registries are a critical component of containerized infrastructure, understanding how they work and how to secure them is crucial for maintaining the integrity and security of containerized applications and their underlying infrastructure.

Figure 4. Distribution of misconfigured, publicly accessible, customer-deployed and –run repositories according to cloud service provider

The state of container image registries

We conducted research on publicly exposed registries and found a significant number of these hosted on Amazon Web Services (AWS). We were able to both connect to these and download their contents without using authentication or brute-force methods.

Here are some key initial data points:

• 9.31 TB worth of downloaded images
• 197 dumped unique registries
• 20,503 dumped images

These figures represent a significant amount of data and a sizeable number of dumped unique registries that could potentially be accessed by malicious actors, highlighting the widespread nature of this issue.

We also discovered that the ownership of these exposed registries was not limited to individual users and smaller companies alone. Rather, some were owned by private organizations with no other public container or code-sharing projects.

The risks of publicly exposed registries

Figure 5. Exposed container registry trends for the past two years based on Shodan data

Exposed registries pose a significant security risk given that without proper authentication mechanisms in place, anyone can pull images from them. This could lead to unauthorized access to proprietary software or sensitive data embedded in the images. Furthermore, if a malicious actor were to gain access to an exposed registry, they could potentially push compromised images back to the misconfigured registry, leading to the spread of malware.

The risks associated with exposed registries are not just theoretical — there have been numerous instances where exposed registries have led to security breaches.

To fully understand the implications of our findings, let's take a deeper look at how container image registries work and why they are crucial in a containerized environment.

The impact of exposed registries

The exposed registries we discovered during our research represent a significant security risk. Without proper authentication, these registries are exposed to anyone who discovers them. This lack of security not only exposes the images stored in the registries but also leaves the door open for anyone to push images to them if write permissions are granted.

The potential impact of this unrestricted access is twofold, with both scenarios being a cause for concern. First, any proprietary or sensitive data stored within the images can be exposed. This could include a wide range of sensitive information, from source code to secrets that do not belong in source code and container images such as API keys and database credentials. In the wrong hands, this information could be used to gain unauthorized access to systems, steal intellectual property, or even launch a targeted attack.

Second, a malicious actor with write access to a publicly exposed registry could push a compromised image to it. This compromised image could contain malicious code, such as a backdoor, a cryptocurrency miner, or even ransomware. Any system that subsequently pulls this image for use would then potentially be running malicious code, effectively spreading the threat across the infrastructure. This could lead to a variety of dangerous scenarios, from data theft and system disruption to a full-scale ransomware attack.

Furthermore, the impact of exposed and misconfigured registries extends beyond the immediate threat to the organizations that own them. Given the interconnected nature of today's software supply chains, a compromised image in one place could quickly spread to other systems and organizations. This ripple effect can amplify the impact of a single exposed registry, turning a localized threat into a widespread security incident.

Securing container image registries

The potential impact of exposed registries on an organization can be far-reaching and could pose a serious threat to the security of containerized applications. This underscores the importance of proper registry security measures, including robust authentication mechanisms, regular vulnerability scanning, and diligent management of permissions.

In this section, we’ll take a closer look at the best practices needed to secure container image registries.

Implementing authentication and identity life cycle processes

The use of authentication measures ensures that only authorized users can access the registry. This can be achieved through various methods, such as traditional usernames and passwords, API keys, or digital certificates. A robust authentication implementation is the first line of defense against unauthorized access to an organization’s registry. Organizations should also ensure that a proper identity life cycle management for users is in place, from onboarding (creating a new digital identity and providing access to resources) to offboarding (ending access to resources when the user leaves the organization).

Organizations and individual users should also follow the principle of least privilege as a fundamental security practice. This principle states that users should be given only the necessary levels of access or permissions they need to perform their work. In the context of a registry, this could mean limiting who can push or pull images, or who can delete or modify existing images.

Scanning images for vulnerabilities

Regularly scanning images for vulnerabilities can help detect known vulnerabilities in the software dependencies included in images. There are various tools available that can automate this process, scanning images whenever they are pushed to the registry and alerting users of any potential issues. This includes not only vulnerabilities, but also hard-coded sensitive data that could be leaked along with the image.

Encrypting data at rest and in transit

By encrypting your data at rest and in transit, users can ensure that even if someone were to gain unauthorized access to their data, they would not be able to read it. There are various methods and tools available for encrypting data, and the best choice depends on the user’s specific needs and the overall environment.

Regularly updating and patching the registry

Just like the upkeep necessary for other types of software, regularly updating and patching container image registries and their underlying software is essential. Implementing updates and patches at regular intervals not only offers new features but also provides fixes for exploitable vulnerabilities and other security issues while ensuring that these registries are equipped with the latest safety measures.

Managed container registries provide another option for organizations. With these kinds of services, the CSP takes care of the patching and underlying configuration, leaving the customer to focus on configuring the access control.

Implementing firewall rules

Regardless of whether organizations are using the container registry for workloads or native services, proper security measures should be applied, observed, and implemented. In the case of container image registries, it is essential to establish firewall rules to block any unwanted connections to the service, configure authentication settings, enable encryption, and continuously monitor the system.

Providing a managed source for internal teams where trusted containers can be pulled from

Providing centralized services simplifies the use of frequently needed functionalities. By logically centralizing the sources for these components and their dependencies, this arrangement provides a means for security teams to assess component properties before integration. Furthermore, this setup diminishes the chance of unexpected issues such as those that arise from changes to existing packages.

Using secretless containers

Secretless containers offer a more secure and efficient method of handling sensitive information within containerized environments. By abstracting the management of secrets away from applications, secretless containers reduce the risk of unauthorized access to critical data, enhancing overall security and mitigating the risk of credential leak even upon a successful registry breach.

Conclusion

The protection of container image registries is a critical aspect of container security, and it is crucial to the entire DevOps environment and its associated infrastructure. As we’ve learned, many developers and other users leave their registries exposed, making them vulnerable to potential threats. By understanding these threats and following the best practices outlined in this post, both individual users and organizations can minimize the risk of a security breach and ensure their container images are safe.

As to why so many registries are left exposed without any security measures, it is difficult to find just one answer. From what we’ve observed in the past, however, misconfigurations play a major role in this issue. Although this might seem like a minor security issue when compared to other types of security lapses, the issue of misconfigurations should be treated with the same concern, especially in an age where cloud technologies are widely used.

Security in the cloud is typically subject to the AWS Shared Responsibility Model, which promotes a cooperative approach by leveraging the strengths of the cloud provider (via infrastructure and technologies) while also relying on the user to build a robust and secure cloud environment (which includes proper configuration of cloud assets). In short, this model relies on both parties collaborating effectively to maintain a safe and reliable cloud infrastructure.

When implementing a private registry, users should ask themselves the following security questions:

• Is it necessary to deploy a private registry service?
• Should this service be on-premises or in the cloud?
• If it should be in the cloud, does the organization understand the trade-offs between a managed service and assuming full responsibility for operating a registry?
• Who can access the registries and how are they being accessed? (That is, via an internal network, a remote network with firewall rules allowing access, or via a remote network with virtual private network [VPN] access?)
• Will the containers hold sensitive or proprietary data? If yes, would encrypting such data have a negative impact?
• Do the container images contain unnecessary secrets?

In a future blog entry, we will explore the specifics of the data we were able to access from these exposed registries. We will also discuss more advanced security measures that interested users and CSPs can implement to further secure container image registries.