🔐☸ Securing apps in Kubernetes

Previously I talk about using ”IT-Tools” to stop data leaks in Engineering teams… now I’ll demonstrate the steps an Engineer can take to security harden a workload in Kubernetes.. specifically a third-party web application for which IT-Tools fits quite nicely, and it can be quite common to deploy third-party apps.

security.md

DevOps today usually includes security in some form aka Devsecops. All the benefits of DevOps but it encompasses security as a integral part of the process.

Prior to all this, security was a bit of a afterthought in the engineering processes. Issues were addressed reactively. Agile focused on speed and collaboration and didn’t fully integrate security or know how to.

It seems obvious but only recently have wider IT governance, risk, and compliance frameworks begun to recognize the importance of integrating security in the software lifecycles. Some even mention DevSecOps by name e..g NIST Cybersecurity Framework (CSF), ISO/IEC 27001 (e.g. principle of secure coding), upcoming PCI DSS 4.0.

…

Back to our app: IT-Tools… it’s an open source web application. We can’t exactly apply all DevSecOps principles can we?

But we can do things to further reduce risks/attack surface. A nice list of guides to help can be found here https://github.com/infoslack/awesome-web-hacking

If you don’t have have prior web technology experience.. try having a speaking to a devs who often possess a deeper understanding of web technologies and can be helpful at guiding what controls to apply instead of throwing things against a wall.

After looking through how IT-Tools works and setting up a test environment. Here’s a list of things I found suitable:

1. Content Security Policy (CSP)

Content Security Policy was introduced back in 2012, surprisingly it’s still a unused tool.

It was introduced to curb vulnerabilities like XSS. Without it an attacker could execute/load arbitrary scripts and to give them a even greater attack surface.

My favorite use of XSS is to replace the page with a login screen. Imagine being sent a legitimate link to tools.yourorg.com, seeing a login screen and thinking nothing of it. Made worse if your password manager has auto-complete enabled.

XSS opens so many possibilities.. imagine..

• You could exploit Same Origin Policy to other sub-domains
• Trick the user to download a file originating from the tools.yourorg.com domain (Using JS)
• Steal Cookies or other information if the application was more complex
• so many other possibilities

A good Security Engineer’s will know how to configure CSP to mitigate problems like these… all while preserving core functionality. Here’s a nice generator tool you can use to help you https://report-uri.com/home/generate.

Content-Security-Policy: default-src 'self'

i.e. This policy tells the browser, anything via self can be loaded. Anything outside of this rule gets blocked by the browser, again if there’s a XSS this will limit what is possible.

2. Kubernetes NetworkPolicy

Looking at the Dockerfile, (https://github.com/CorentinTh/it-tools/blob/main/Dockerfile) indicates the application is nothing more than a static website.

It’s base image is nginx, naturally the pod should never initiate an external connection under normal circumstances. We’ll use NetworkPolicy rule to enforce this.

3. Build from source & version pinning

Like I mentioned above, the apps been developed already.

This might presents a additional risk.. third-party apps pose a attractive target to attackers; it’s a easy way into many organizations at once.

Some open-source projects can be modified by anybody, or maintainer can be manipulated financially and inject malicious code. Not to mention the thousands of dependencies a typical app utilizes these days - where we’ve previously seen supply chain attack.

The NCSC has a nice guide on supply chain security: https://www.ncsc.gov.uk/collection/supply-chain-security and explains the various attacks possible.

Is it an impossible task to guard against? No, tthere are some things we can do to mitigate some of the risks posed here.

Things we can do

1. Static Application Security Testing (SAST)
2. Dynamic Application Security Testing (DAST)
3. Interactive Application Security Testing (IAST)
4. Software Composition Analysis (SCA)
5. Pin versions

We have to trust stuff at some stage. I think a pragmatic approach would be to build our own container image from a commit hash - pinning it’s code and dependencies (like package-lock.json).

4. Seccomp

Seccomp stands for Secure Computing and integral to Linux containerization, together with Namespaces, cgroups and SELinux. It is a Linux feature used to restrict the set of system calls that an application are allowed.

Docker (although not k8s default container runtime) disables around 44 syscalls (over 300 are available). Each runtime have a default Seccomp profile which can differ. Default profiles are pretty easy to find and useful when determining your workloads compatibility.

• containerd (https://github.com/containerd/containerd/blob/f0a32c66dad1e9de716c9960af806105d691cd78/contrib/seccomp/seccomp_default.go#L51)
• cri-o (https://github.com/cri-o/cri-o/blob/main/internal/config/seccomp/seccomp.go)
• Docker (https://github.com/microsoft/docker/blob/master/docs/security/seccomp.md)

Interestingly default policies are not enabled by default in Kubernetes however you can easily set this up after testing compatibility with your workloads. (Requires v1.27+)

5. AppArmour

ChatGPT says: “AppArmor is a crucial component of Linux security, providing mandatory access control (MAC) by confining individual programs within defined security profiles.

Profiles define what actions an application can perform, including file access, network communication, and system calls.”

Nice so we can find/create a custom AppArmor profile tailored specifically for the nginx container. This profile would define strict rules governing the nginx process’s behavior, ensuring that it only accesses the necessary files and resources required to serve static web content.

here’s a simplified example of an AppArmor profile for an nginx container:

apparmor

#include <tunables/global>

profile nginx_profile {
  # Allow read access to static web content
  /var/www/html/** r,

  # Allow read access to nginx configuration files
  /etc/nginx/nginx.conf r,
  /etc/nginx/conf.d/** r,

  # Allow network access for serving HTTP traffic
  network inet tcp,

  # Allow DNS resolution for hostname resolution
  network inet domain,

  # Allow access to necessary system libraries
  /usr/lib/** mr,

  # Allow access to essential system resources
  capability sys_chroot,
  capability setgid,
  capability setuid,

  # Deny access to potentially dangerous capabilities
  deny capability dac_override,
  deny capability dac_read_search,
  deny capability fowner,
  deny capability fsetid,
  deny capability kill,
  deny capability mknod,
  deny capability sys_admin,
  deny capability sys_boot,
  deny capability sys_module,
  deny capability sys_ptrace,
  deny capability sys_rawio,
  deny capability sys_time,
  deny capability sys_tty_config
}

It allows read access to the nginx configuration files (/etc/nginx/nginx.conf and /etc/nginx/conf.d/) and the static web content directory (/var/www/html/). It also permits network access for serving HTTP traffic and DNS resolution.

Also granting access to essential system libraries (/usr/lib/**) and capabilities necessary for nginx operation, such as sys_chroot, setgid, and setuid. Meanwhile, it explicitly denies access to potentially dangerous capabilities that are unnecessary for nginx, such as sys_admin, sys_rawio, and sys_ptrace.

6. No root privileges

Remember the principle of least privilege? It ensures containers run with the minimum permissions necessary to perform their intended tasks

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx-container
    image: nginx:latest
    securityContext:
      runAsNonRoot: true
      runAsUser: 1000
      readOnlyRootFilesystem: true
    ports:
    - containerPort: 80

Here we have a Kubernetes pod (nginx-pod) running a single container (nginx-container) based on the nginx:latest image… the application doesn’t need root so the container doesn’t get r00t:

runAsNonRoot: true: This setting ensures that the nginx container runs as a non-root user, rather than with root privileges. By running as a non-root user, the container’s access to system resources and potentially sensitive files is restricted, reducing the impact of any security vulnerabilities or exploits.

runAsUser: 1000: We specify a specific non-root user ID (UID) for the container to run as. This further limits the container’s access rights and ensures that it operates within a confined environment.

readOnlyRootFilesystem: true: Setting the root filesystem to read-only prevents any writes to the root filesystem within the container. This adds an extra layer of security by mitigating the risk of unauthorized modifications to critical system files or configurations.

7. TLS

Our app might be served to our internal users and they will use a VPN. So we don’t necessarily need TLS.

But imagine a scenario where the site is served publicly.. TLS becomes necessary to prevent MITM attacks… Someone in the middle like a rogue ISP can snoop and also modify the contents of the page dynamically… who remembers ISPs injecting ads into our pages in 2000’s?

I will skip setting up TLS here, you can override nginx’s default config or setup a sidecar container to serve the app with TLS.

Summary

In another post I will demo and publish a repo with all the mentioned rules in a live environment.