<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=521127644762074&amp;ev=PageView&amp;noscript=1">

2023 Benchmark Kubernetes Report: The State of Kubernetes Workload Security

Moving to the cloud remains an important business initiative for organizations around the world. According to the  World Economic Forum, digital transformation can enable sustainable growth and innovation. Part of the move forward with digital transformation plans includes moving more applications and services to the cloud. However, managing security presents new challenges for organizations adopting cloud and cloud-native technologies such as Kubernetes. According to Red Hat’s  2022 State of Kubernetes security report, 93% of respondents experienced at least one security incident in their Kubernetes environments. As organizations increasingly move production workloads to Kubernetes, it is important to understand both how to secure all aspects of Kubernetes and track and monitor workload security over time.

Fairwinds gathered data from over 150,000 workloads and hundreds of organizations to assemble the 2023 Kubernetes Benchmark Report, analyze trends in 2022, and compare it to data from the previous year. While a recent CNCF report indicated that 96% of respondents were using or evaluating Kubernetes, aligning to best practices can be difficult for organizations of all sizes. Unfortunately, the lack of alignment comes with real consequences: elevated security risks, unmanaged cloud costs, and decreased reliability of cloud apps and services.

Insecure Capabilities

Kubernetes is not secure by default, but many developers are not aware of this. For example, some Linux capabilities are enabled by default for Kubernetes workloads, even though most workloads do not actually require those capabilities. Unfortunately, the latest benchmark data shows that organizations are not limiting these capabilities as much in 2022 as they did the previous year. In 2021, 42% of organizations turned off these capabilities for most workloads (only 0-10% of workloads were impacted). In 2022, only 10% of organizations had these same  insecure capabilities turned off.  

“33% of organizations have more than 90% of workloads running with insecure capabilities, an increase compared to the previous year.” 

Writeable File Systems

The security setting readOnlyRootFilesystem prevents a container from  writing to its filesystem. It is important to enable this setting in case an organization is hacked. This setting can ensure that if an attacker gets in, they will be unable to tamper with the application or write foreign executables to disk. This is another instance where it is not set to true by default on Kubernetes workloads, which means teams must explicitly change the setting to ensure the most secure configuration possible. In the past, only 23% of organizations appeared to be unaware that they needed to change this setting to override the insecure default setting for 71%-100% of their workloads. Unfortunately, that number increased in 2022; now 56% of organizations are failing to make that override. As Kubernetes usage and adoption grows, it is alarming to see this trend in this direction.

Privilege Escalation Allowed 

Under some configurations, containers may have the ability to  escalate its privileges. If you set allowPrivilegeEscalation to false, that sets the no_new_privs flag on the container process, which prevents setuid binaries from changing the effective user ID. It is especially important to set this flag when you are using runAsNonRoot, which can otherwise be circumvented. This security is also not set by default, which means that security-conscious teams must explicitly set it. This year, the benchmark report shows a disturbing increase in workloads open to privilege escalation. In 2021, 42% of organizations locked down the majority of workloads. In 2022, that number dropped to 10%. 

Runs as Privileged 

The privileged command determines whether any container in a pod can enable  privileged mode. By default, a container may not access any devices on the host. However, a privileged container has access to all devices on the host. When this feature is enabled, it allows the container nearly the same level of access as processes running on the host. This is useful for containers that need to use Linux capabilities, such as manipulating the network stack and accessing devices. In this case, the privileged flag is off by default. Likely for this reason, 87% of organizations have the privileged flag off, which increases the security of workloads. In 2021, 88% of organizations had the privileged flag off, so it decreased slightly in 2022. 

Run as Root Allowed

Another insecure capability is  running containers as root. Many workloads in the  Benchmark Report, unfortunately, are allowed this capability. The benchmark data shows an increase in the number of workloads in which running as root is allowed. In 2022, 44% of organizations were running 71% or more of their workloads allowing root access. That is an increase of twenty-two points compared to 2022. Given that there are known vulnerabilities targeting this capability, it is an alarming increase to see in the report. 

Image Vulnerability

Workloads impacted by  image vulnerabilities increased significantly in 2022. In 2021, 40% of organizations had fewer than 10% of workloads impacted by image vulnerabilities; in 2022, that fell to only 12% of organizations. Malicious actors exploit known vulnerabilities, therefore they must be patched or remediated as quickly as possible. According to the report, 62% of organizations have more than 50% of workloads impacted by vulnerabilities. 

Is your organization running images with vulnerabilities?

Outdated Helm Charts 

Keeping up to date with the latest releases for all your cluster add-ons is challenging, so it’s not surprising that  outdated Helm charts are a common issue across most organizations. In 2022, 46% of organizations have 50% or greater workloads impacted by running workloads from outdated Helm charts. The add-ons running your cluster are probably installed by Helm. Each add-on has its own release cadence, and some updates include critical security patches. Helm charts must be kept up to date, but they can be difficult to monitor and predict.  Nova is an open source project that cross checks Helm charts running in the cluster with the latest version available, which makes it easy to see when updates are available.

Outdated Container Images 

This year we started benchmarking how many organizations are running  outdated container images. The new benchmark data shows that either less than 10% or greater than 90% of workloads are impacted. Nova, an open source tool, can run a flag called “containers” to analyze all container images in a Kubernetes cluster and notify users if an updated version is available. Nova provides three alternatives for updating images: 

  • the latest version

  • the latest minor version

  • the most recent patch version

This allows users to choose the patch that they feel most comfortable using for the specific container.

API Version Deprecated 

According to the benchmark data, most organizations have just a few workloads with deprecated API versions. Unfortunately, in this category it is also trending in the wrong direction. In 2021, 82% of organizations were up to date with API versions for the vast majority of their workloads. In 2022, that number decreased to 74%. Monitoring for deprecated APIs and keeping them up to date remains a critical step in reducing risk during Kubernetes upgrades.

Kubernetes Maturing, but Security Configurations Remain Challenging

Using containers and Kubernetes for container orchestration enables a shift to cloud-native applications and services. These infrastructure changes bring significant value to organizations today. However, as many rapidly adopt Kubernetes and endeavor to deploy  more and more applications to Kubernetes, they must also understand the many configurations available and how to set them appropriately. The Kubernetes Benchmark report can help you understand both where configurations are deficient or trending in the wrong direction and how to make changes going forward to ensure that your organization’s deployment is as secure, reliable, and cost-efficient as possible. 

Read  the complete Kubernetes Benchmark Report today.