Container Security: Hardening Docker & Kubernetes

Containers revolutionized how we deploy applications, but they also introduced new attack vectors. Here’s how to harden your containerized infrastructure based on real-world production experience.

The Container Security Model

Container security spans multiple layers:

• Image Security - Base images, dependencies, secrets
• Runtime Security - Process isolation, resource limits
• Network Security - Segmentation, encryption
• Orchestration Security - RBAC, admission controllers

Docker Security Fundamentals

Secure Base Images

# ❌ Vulnerable - outdated, bloated base image
FROM ubuntu:18.04

# ✅ Secure - minimal, updated base image
FROM alpine:3.19

# Even better - distroless for production
FROM gcr.io/distroless/nodejs:18

# Multi-stage build for minimal surface area
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

FROM gcr.io/distroless/nodejs:18
COPY --from=builder /app/node_modules ./node_modules
COPY . .
USER 1001
EXPOSE 3000
CMD ["server.js"]

Non-Root Containers

# Create dedicated user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

# Set ownership of app directory
COPY --chown=nodejs:nodejs . /app

# Switch to non-root user
USER nodejs

# Verify user
RUN whoami # Should output: nodejs

Image Scanning & Vulnerability Management

# Scan images before deployment
docker scout cves my-app:latest

# Trivy scanning
trivy image --severity HIGH,CRITICAL my-app:latest

# Snyk container scanning
snyk container test my-app:latest

Docker Daemon Security

{
  "icc": false,
  "userns-remap": "default",
  "log-driver": "syslog",
  "log-opts": {
    "syslog-address": "tcp://log-server:514"
  },
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Hard": 64000,
      "Soft": 64000
    }
  }
}

Kubernetes Security Deep Dive

Pod Security Standards

# Pod Security Policy (deprecated but important concept)
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - "configMap"
    - "emptyDir"
    - "projected"
    - "secret"
    - "downwardAPI"
    - "persistentVolumeClaim"
  runAsUser:
    rule: "MustRunAsNonRoot"
  seLinux:
    rule: "RunAsAny"
  fsGroup:
    rule: "RunAsAny"

Security Contexts

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1001
    runAsGroup: 1001
    fsGroup: 1001
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: my-app:latest
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop:
            - ALL
          add:
            - NET_BIND_SERVICE
      resources:
        limits:
          memory: "512Mi"
          cpu: "500m"
        requests:
          memory: "256Mi"
          cpu: "250m"
      volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: var-run
          mountPath: /var/run
  volumes:
    - name: tmp
      emptyDir: {}
    - name: var-run
      emptyDir: {}

RBAC Configuration

# Service Account
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-service-account
  namespace: production

---
# Role with minimal permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: app-role
rules:
  - apiGroups: [""]
    resources: ["configmaps", "secrets"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]

---
# Role Binding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-role-binding
  namespace: production
subjects:
  - kind: ServiceAccount
    name: app-service-account
    namespace: production
roleRef:
  kind: Role
  name: app-role
  apiGroup: rbac.authorization.k8s.io

Network Policies

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-app-traffic
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: my-app
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: ingress-nginx
      ports:
        - protocol: TCP
          port: 3000
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              name: database
      ports:
        - protocol: TCP
          port: 5432
    - to: {} # Allow DNS
      ports:
        - protocol: UDP
          port: 53

Secrets Management

External Secrets Operator

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
  namespace: production
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "app-role"

---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: app-secrets
  namespace: production
spec:
  refreshInterval: "10m"
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: app-secret
    creationPolicy: Owner
  data:
    - secretKey: database-password
      remoteRef:
        key: database
        property: password

Sealed Secrets

# Install kubeseal
curl -OL https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.18.0/kubeseal-0.18.0-linux-amd64.tar.gz

# Create sealed secret
echo -n mypassword | kubectl create secret generic mysecret --dry-run=client --from-file=password=/dev/stdin -o yaml | kubeseal -f - -w mysealedsecret.yaml

Runtime Security

Falco for Runtime Detection

# Custom Falco rule
- rule: Shell spawned in container
  desc: A shell was spawned in a container
  condition: >
    spawned_process and
    container and
    shell_procs and
    proc.pname exists and
    not proc.pname in (shell_binaries)
  output: >
    Shell spawned in container
    (user=%user.name container_id=%container.id container_name=%container.name
     shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)
  priority: WARNING
  tags: [container, shell, mitre_execution]

OPA Gatekeeper Policies

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredsecuritycontext
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredSecurityContext
      validation:
        openAPIV3Schema:
          type: object
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredsecuritycontext

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not container.securityContext.runAsNonRoot
          msg := "Container must run as non-root user"
        }

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          container.securityContext.allowPrivilegeEscalation != false
          msg := "Container must not allow privilege escalation"
        }

---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredSecurityContext
metadata:
  name: must-run-as-nonroot
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    namespaces: ["production"]

Image Supply Chain Security

Sigstore/Cosign Integration

# Sign container image
cosign sign --key cosign.key my-registry/my-app:v1.0.0

# Verify signature
cosign verify --key cosign.pub my-registry/my-app:v1.0.0

# Admission controller to enforce signed images
apiVersion: v1
kind: ConfigMap
metadata:
  name: cosign-verification
data:
  policy.yaml: |
    apiVersion: kyverno.io/v1
    kind: ClusterPolicy
    metadata:
      name: verify-image
    spec:
      validationFailureAction: enforce
      background: false
      rules:
      - name: verify-signature
        match:
          any:
          - resources:
              kinds:
              - Pod
        verifyImages:
        - image: "my-registry/*"
          key: "cosign.pub"

Monitoring & Compliance

Security Metrics Dashboard

# Prometheus rules for security metrics
groups:
  - name: security
    rules:
      - alert: UnauthorizedAPIAccess
        expr: increase(apiserver_audit_total[5m]) > 10
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "High number of unauthorized API access attempts"

      - alert: PrivilegedContainerDetected
        expr: kube_pod_container_status_running{container!=""} and on(pod) kube_pod_spec_containers_security_context_privileged == 1
        for: 0m
        labels:
          severity: critical
        annotations:
          summary: "Privileged container detected in {{ $labels.namespace }}/{{ $labels.pod }}"

Security Scanning Pipeline

# GitHub Actions security pipeline
name: Container Security Scan
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build -t test-image .

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: "test-image"
          format: "sarif"
          output: "trivy-results.sarif"

      - name: Upload Trivy scan results
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: "trivy-results.sarif"

      - name: Hadolint Dockerfile
        uses: hadolint/hadolint-action@v2.0.0
        with:
          dockerfile: Dockerfile
          failure-threshold: error

Key Security Checklist

Docker Security

• ✅ Use minimal base images
• ✅ Run as non-root user
• ✅ Enable content trust
• ✅ Scan images for vulnerabilities
• ✅ Use multi-stage builds
• ✅ Limit container resources

Kubernetes Security

• ✅ Enable RBAC
• ✅ Use network policies
• ✅ Implement pod security standards
• ✅ Secure etcd with encryption
• ✅ Regular security updates
• ✅ Audit logging enabled

Runtime Security

• ✅ Monitor with Falco
• ✅ Use admission controllers
• ✅ Implement OPA policies
• ✅ Regular vulnerability scanning
• ✅ Incident response plan

Container security is an ongoing journey, not a destination. These practices have helped me secure production Kubernetes clusters handling millions of requests daily.

What container security challenges are you facing? Let’s discuss on LinkedIn.

🔗 Read more

• API Design Patterns - Building APIs that don’t suck