Cloud
7 Container Security Best Practices For Better Apps
Explore how to implement 7 container security best practices within a CI/CD pipeline built with tools from Amazon Web Services (AWS).
What is container security?
Container security is the protection of a containerized application by implementing and maintaining security controls that protect containers and the underlying infrastructure. This doesn’t just mean securing the container host and images, but the entire component stack used throughout the DevOps lifecycle.
When securing containers, the seven main concerns are:
- Security of container host
- Container network traffic
- Security of application within your app
- Malicious behavior within your application
- Securing your container management stack
- Foundation layers of your app
- Integrity of the build pipeline
Notice how we said “DevOps lifecycle” instead of developer lifecycle? That’s because secure apps can’t happen without both teams working together. Development and security operation teams must assume responsibility during specific phases: source, build, test, deploy, and monitor. More on that, later.
We will review common container security concerns and challenges, as well as seven best practices to help secure your containers without disrupting development workflows.
Six container security challenges
With 92% of containers being used in production, ensuring they’re secure is vital to protecting your app and organization from a breach. So, what are the potential roadblocks standing between you and secure containers? Here are six reasons container security can be challenging:
- Container environment exposed to the outside world: The majority of the time, containers are created to be publicly accessible. Leaving insecure containers accessible to the public is like putting out a rotten banana on a hot summer day—the flies don’t miss the opportunity to pounce.
- Rampant use of open source code: While convenient, unchecked open source code poses many security threats. There’s no secrets validation from open source code, which can exposure your organization. For example, the code contains the IP address of the contributing organization, so when the code is running, there will be an occasional remote procedure call (RPC) for that IP address, which will be dead ended because it doesn’t exist within your organization. However, attackers are monitoring and listening for those rogue RPC; they try to mimic the home the IP address is calling to gain access to your cloud environment.
- Need for speed: As the time to market continues to shrink, developers are pressed to move faster and deliver more value, which leads to teams often overlooking security to meet deadlines. But do developers really need to skip security to deploy on time? No. It’s just a common attitude that security slows building down. In fact, implementing security throughout the build lifecycle can save developers time over the long run.
- Diversity of tools:There’s an endless buffet of tools and services for developers to use, making it complex for organizations to keep track of what needs to be updated and patched. This lack of visibility can leave the software supply chain vulnerable to attacks.
- Lack of compliance checks for development procedures: Since the development environment is very fast and constantly evolving, it can be difficult to implement compliance checks and ensure you’re adhering to all the right rules.
- Responsibilities are unclear: Since containers can run in any cloud environment (public, private, or even a combination) as well as on-premises, there’s often confusion around who is responsible for what during different development phases. The lack of clear structure leads to security gaps such as overly permissive privileges, exposed secrets and keys, and more.
Your natural reaction may be to close this article because you don’t need this negativity in your life. Please don’t. Since we’ve reviewed the challenges, it’s time to explore seven container security best practices that won’t slow down your workflows or add to your workload.
Seven container security best practices
By following these best practices, you can take your barebones pipeline from this:
To this, a security dream leveraging automation and customizable APIs so you can build quickly and securely while appeasing security operations teams.
- Write clean code
In 2020, Digital Shadow scanned more than 150 million entities from GitHub, GitLab, and Pastebin and found 800,000 access keys and secrets. 40% of these were for database stores—38% for CSPs such as Google, Microsoft Azure, and AWS. Yikes.
It goes without saying (but I’m saying it anyways) you cannot afford to have your organization’s secrets exposed. To keep your secrets secret, avoid hardcoding secrets into the code or in a config file that is pushed into a repository. You can use tools like Git Secrets to prevent you from committing passwords and other sensitive information to a Git repository. Using Git Secrets or another secrets manager is a best practice.
Lastly, we recommend leveraging a tool like Amazon CodeGuru Reviewer to inspect your code as your write and notify you of any potential vulnerabilities. The earlier vulnerabilities are detected and mitigated, the less stress (and cost) down the road. - Use hardened container images
Hardening helps limit potential weakness and reduce vulnerabilities by analyzing container images’ current security status and then making any necessary improvements. To simplify the process you can leverage hardened images for container operating systems from organizations like the Center for Internet Security (CIS). However, don’t just blindly trust these pre-hardened images—make sure they’re continuously scanned for any vulnerabilities that may have snuck in along the way.
By building pipelines with hardened images, you’re creating standardized base images for all internal teams to use. Scanning base images as your build gives you insights into your evolving security posture and alert you of any new vulnerabilities that may emerge. A reoccurring theme—the earlier you catch a potential problem, the better. - Secure your images
According to Sysdig, 40% of images are pulled from public sources, which can be troubling considering Docker Hub only certifies less than 1% of its millions of hosted images. To secure your image, make sure you only pull images from trusted sources and store them in your own private and secure repository.
Private repositories provide the necessary control for proper access management. Make sure only those who need access to complete their job. Do not run your images as root—this is overly permissive and could allow bad actors to inject malicious code if they obtain access.
Similar concepts apply to securing your artifacts: use a private repo for storage and only use packages validated by the security team. - Test your containers throughout the pipeline
Use different testing methods throughout the pipeline, instead of leaving testing to the end.
The first testing method is container image scanning, which helps identify software vulnerabilities. Next, use static application security testing (SAST) tools to analyze source or compiled versions of code to help you find security flaws. Lastly, dynamic application security testing (DAST) tools auto-scan web applications from the outside to look for security vulnerabilities like SQL injection, command injection, or insecure server configuration. DAST is usually completed after the application has been deployed to a staging environment.
Ideally, your images should be scanned inline to control your privacy in case they contain credentials by mistake. - Manage secrets securely
As we mentioned, you shouldn’t store secrets in code. So where should you keep them? Utilize a dedicated secrets manager, like AWS Secrets Manager, and regularly rotate your secrets. Your application should be configured to call the secrets manager and access the appropriate secret only when it needs them. - Ensure observability in production
Security operation teams need the entire picture to mitigate threats as early as possible. This is why collaboration is so important. Without help from SecOps, the build process could be stalled due to undetected vulnerabilities.
Since containers are ephemeral, meaning they’re quickly spun up and destroyed, it can be difficult for security teams to monitor and track changes, especially in complex systems with high churn.
Containers also share resources like memory and CPU across one or more hosts, making it challenging to monitor resource consumption on the physical host and get an accurate indication of the container performance or application health.
To help SecOps help you, look into security tools that provide comprehensive visibility without interfering with your job. The right tool should provide sufficient insights into the metrics and logs needed to properly monitor and measure container health.
Don’t forget to observe the network as well and make sure you’re receiving security alerts so next steps can be taken. - Remember to secure the pipeline
We touched on this briefly before, but access management is extremely important. Follow the concept of least privilege for IAM permissions and roles attached to the pipeline and its components. If someone doesn’t need access to complete a task, don’t give it to them. You should take a zero trust approach to this as well, wherein you never trust and always verify any devices, applications, or users requesting access.
Using infrastructure as code (IaC) is a great way to ensure your app containers are secure when deployed. Just remember to take the necessary security processes here like scanning your IaC templates before deploying, in the event configurations were changed by other teams.
Next steps
Container security can be complex but leveraging the right security tools to accomplish each best practice is an effective, low-hassle method. There are several tools out there, but we recommend a platform approach with automation—ensuring total visibility for SecOps teams and minimum workflow interrupts for DevOps teams.
Check out these resources for more insights into container security:
[Documentation] Trend Micro Cloud One™ – Container Security
[Video] Seven Things DevOps Needs to Know About Container Security
[Blog] How to use Rancher in Kubernetes
[Blog] Detect Container Drift in Your Kubernetes Deployments
