Seven Guidelines for Implementing Zero Trust in Kubernetes



In the wake of a seemingly endless parade of catastrophic security breaches and ransomware attacks, the Biden Administration hit the gas pedal in May 2021 with an executive order mandating improvements to the nation’s security technology and specifically calling out the need for zero trust (ZT) security models.

The National Institute of Standards and Technology (NIST) followed up in August with a whitepaper defining Zero Trust Architecture (ZTA) and exploring “deployment models and use cases where ZT could improve an enterprise’s overall information technology security posture”. Various government agencies, including the Cybersecurity and Infrastructure Security Agency (CISA) and the Office of Management and Budget, are releasing documents to guide ZT implementation, including a maturity model to help implementers understand the journey to full ZT deployment.

The Kubernetes community has been discussing ZT for number of years as a key component of an end-to-end encryption strategy. Service mesh providers have baked in some key practices (such as mTLS and certificate‑key rotation) in an effort to make it easier to implement ZT, but that said, we are still in the early days of implementing robust ZTAs in applications at scale. There remains some debate over what ZT means for Kubernetes and what the best practices are. Out of the box, Kubernetes presents significant security challenges to teams that aren’t educated on how Kubernetes networking and security differ from traditional IT and infrastructure systems.

What Is Zero Trust and Why Does It Matter?

Traditional security models establish a strong perimeter around a deployment environment, with a centralized gatekeeper verifying user identity and allowing only authorized users to access the infrastructure inside. With adoption of microservices, clouds, and distributed deployments, this model is no longer an option because it’s increasingly unclear where the perimeter is or if there even is one. Enter ZT.

In a ZT model, quite literally nothing and no one is trusted – users, applications, networks, servers, services, nor APIs. Every single element at every single layer must be authenticated and tested for authorization. When technology assets, apps, or services connect and exchange data, all communication is routed through a specified agent that authenticates all parties and grants them privileges based on access policy. Importantly, ZT systems operate at every level on a least‑privilege basis, denying access to all parties save those explicitly authorized for a particular resource.

The benefits of ZT are numerous – it improves your security posture by:

  • Automatically preventing unauthorized activity
  • Reducing the accessible attack surface through access controls
  • Quickly detecting behavioral anomalies and indicators of compromise
  • Limiting access time through real‑time least‑privilege policies
  • Making security independent of all other variables, including environment and geography
  • Blocking ongoing attacks through constant authentication and identity validation

ZT is particularly important for cloud‑native apps and infrastructure. Loosely coupled and portable, cloud‑native apps are designed to run in containers and change location and status as needed. Everything is ephemeral except for the code, configuration, and a handful of necessary elements such as IP addresses that link a cloud‑native app to the outside world or to internal services. East‑west traffic (internal to an environment) and north‑south traffic (entering and exiting the environment) look more and more similar. APIs intermediate all communications, both within the app environment and with external clients. Constantly validating privilege and identity is not only useful but, ultimately, a security necessity.

Seven Guidelines for Implementing Zero Trust in Kubernetes

Deploying ZT for Kubernetes‑powered infrastructure and applications can be challenging, so here we offer a set of guidelines. They may seem obvious but can be surprisingly challenging to implement from scratch.

Guideline #1: Avoid Adding Complexity to the Kubernetes Architecture

Running Kubernetes without a ZT framework is challenging, and adding ZT can make things even more complicated. With many vendors, the default way to provide additional services or functionality is with a new sidecar or Kubernetes Custom Resource Definition (CRD). Unfortunately, each such addition increases complexity. For example, when you add a new sidecar to transmit observability data, Kubernetes has to maintain one and possibly two more network connections. Added complexity can slow down application speed and bloat infrastructure costs by requiring larger pods to accommodate application needs. Occam’s razor applies here: the simplest path to ZT with the fewest sidecars and CRDs is usually the best one.

Guideline #2: Add Minimal Overhead to Developers and End Users

Developers don’t want to think about ZT and there’s no reason to make them. When implementing ZT forces developers to change their behavior or workflows, they’re likely to push back because they see security as slowing them down. Changing behavior or workflows also significantly boosts the chance of human error – “wetware” is always a weak link in the security chain. As a NetOps or SecOps engineer, you’ve succeeded if ZT mechanisms are so transparent that developers don’t even know they’re there, and the same goes for end users and customers. In fact, by making security policies stronger and more cleanly automated, ZT in theory can even improve end‑user experience by removing some of the uglier aspects of multi‑factor authentication and the bolted‑on feel of many security processes.

Guideline #3: Apply ZT Rules to Both the Data Plane and the Control Plane

While the data plane is “where the action is” for applications, the control plane is often just as vulnerable – and sometimes even more – to supply chain attacks and other compromises. It takes more work to enforce ZT compliance on the control plane than the data plane because of complex elements like policy engines, data collection systems for telemetry and tracing, and more. It might be tempting to make the data plane ZT and worry less about the control plane with all its moving parts, but you must ensure both are compliant to maximize the impact of ZT.

Guideline #4: Enforce ZT for East‑West and North‑South Traffic

Many organizations opt at first to apply ZT only to north‑south traffic because traffic originating outside the environment seems less trustworthy than internal traffic. This approach is a mistake. In Kubernetes, east‑west traffic looks and behaves much like north‑south traffic. Kubernetes services, nodes, and pods all communicate via API over HTTP or HTTPS. It is significantly more secure to apply the same ZT policies and processes to all traffic and doing so doesn’t create that much more overhead. Also, by applying ZT everywhere from the start you eliminate the risks or headaches of trying to bolt on east‑west ZT after your north‑south ZT practices may have evolved.

Guideline #5: Use Both an Ingress Controller and a Service Mesh

It is quite possible to build and run applications in Kubernetes without either an Ingress controller or a service mesh. But you need them both if you want to make ZT an easy default setting for all elements in your infrastructure. An Ingress controller incorporates some of the more useful features of API gateways and load balancers. It also has the strong benefit of behaving like a true reverse proxy, able to route traffic to specific Kubernetes pods (unlike traditional load balancers). A service mesh radically simplifies both enforcement of ZT on east‑west traffic and observability and reporting for auditing and validation purposes. So if you want to take an easier and cleaner path to ZT, design for both an Ingress controller and a service mesh at the same time.

Guideline #6: Integrate the Ingress Controller with the Service Mesh

This is a close corollary to guideline #5. Not all Ingress controllers can tightly integrate with all service meshes. For example, Istio‑based service meshes use a different type of certificate management system than NGINX Ingress Controller. Ensuring during the design phase that the two tools can tightly integrate with minimal effort avoids considerable complications and configuration hacks later on. For an example of an integrated north‑south and east‑west solution, read How to Simplify Kubernetes Ingress and Egress Traffic Management.

Guideline #7: Automate Proper Handling of Certificates

As for most other secure forms of connectivity, certificate maintenance is crucial to running the public key infrastructure (PKI) component for encryption in Kubernetes. For ZT conformance, certificate management must be automated and dynamic, meaning that certificates expire and rotate out on a regular basis to ensure continuous authentication. Both your service mesh and your Ingress controller need automated certificate issuance, rotation, and expiration. If the native or best‑integrated certificate management tool for an Ingress controller or a service mesh cannot do this, you either need to figure out another option or walk away.

Zero Trust with NGINX: Invisible, Omnipresent, and Easy

We are in the very early days of ZT. If Kubernetes adoption partners are any indication, the learning curve is a little steeper than we would like. That said, the trends towards cloud‑native and cloud‑hosted apps – and continued cybersecurity threats – make it imperative to move towards ZT sooner rather than later. By using common sense and making smart choices about the systems you deploy for ZT, you can transition faster and more smoothly toward the end goal of making ZT not only invaluable but also invisible, omnipresent, and easy.

Enabling ZT in Kubernetes is not a trivial thing, but using an Ingress controller and service mesh with pre‑baked ZTA capabilities saves you time and headaches. You can get started today by using NGINX Ingress Controller with NGINX Service Mesh. With them, just a single CLI command to make a simple config change enables most what you need for ZT.