Skip to content

CKS: Cluster Setup

15% of the CKS exam, 5 competencies. Back to the CKS overview.

Each competency lists what to know and the official documentation to read, then a demo you can run. Demos run on the demo lab, a throwaway kind cluster on your own machine; set it up once before you start.

Use Network security policies to restrict cluster level access

Section titled “Use Network security policies to restrict cluster level access”

Know

  • Default-deny per namespace, explicit allows, and blocking egress to the cloud metadata address (169.254.169.254) unless needed.

Read

Demo: Default deny, DNS allowed, metadata blocked

  1. Create a namespace for this demo and make it the default for your context.

    Terminal window
    kubectl create namespace cks-netpol
    kubectl config set-context --current --namespace=cks-netpol
  2. Lock the namespace down in both directions, allow DNS, and allow web traffic out except the metadata address.

    Terminal window
    kubectl apply -f - <<'EOF'
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
    name: default-deny-all
    spec:
    podSelector: {}
    policyTypes: ["Ingress", "Egress"]
    ---
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
    name: allow-dns
    spec:
    podSelector: {}
    policyTypes: ["Egress"]
    egress:
    - to:
    - namespaceSelector:
    matchLabels:
    kubernetes.io/metadata.name: kube-system
    podSelector:
    matchLabels:
    k8s-app: kube-dns
    ports:
    - {protocol: UDP, port: 53}
    - {protocol: TCP, port: 53}
    ---
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
    name: allow-https-out
    spec:
    podSelector:
    matchLabels:
    egress: internet
    policyTypes: ["Egress"]
    egress:
    - to:
    - ipBlock:
    cidr: 0.0.0.0/0
    except: ["169.254.169.254/32", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
    ports:
    - {protocol: TCP, port: 443}
    EOF
  3. Test from a pod with and without the label.

    Terminal window
    kubectl run locked --image=registry.k8s.io/e2e-test-images/busybox:1.37.0-2 -- sleep 3600
    kubectl run outbound --image=registry.k8s.io/e2e-test-images/busybox:1.37.0-2 --labels=egress=internet -- sleep 3600
    kubectl wait --for=condition=Ready pod/locked pod/outbound --timeout=60s
    for p in locked outbound; do echo "$p:"; kubectl exec $p -- sh -c 'nslookup registry.k8s.io >/dev/null && echo dns ok; nc -zv -w 3 registry.k8s.io 443'; done
    kubectl exec outbound -- sh -c 'nc -zv -w 3 kubernetes.default.svc.cluster.local 443' || true

What you should see: Both pods resolve names. Only outbound opens port 443 on the internet, and even it can’t reach the API server’s service IP, because private ranges are excluded. Policies are additive allows; the deny comes from being selected by any policy at all.

Clean up:

Terminal window
kubectl config set-context --current --namespace=default
kubectl delete namespace cks-netpol

Practise

  • Write a policy that denies all egress except DNS and one internal Service.

On this site

Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)

Section titled “Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)”

Know

  • Run kube-bench, read the failed checks, and fix them in the static pod manifests or kubelet configuration.

Read

Demo: Run kube-bench on the control plane

  1. Create a namespace for this demo and make it the default for your context.

    Terminal window
    kubectl create namespace cks-cis
    kubectl config set-context --current --namespace=cks-cis
  2. Use the job manifest from the pinned kube-bench release, with its image pinned too.

    Terminal window
    curl -sLO https://raw.githubusercontent.com/aquasecurity/kube-bench/v0.16.0/job-master.yaml
    sed -i 's|aquasec/kube-bench:latest|aquasec/kube-bench:v0.16.0|' job-master.yaml
    grep image: job-master.yaml
    kubectl apply -f job-master.yaml
    kubectl wait --for=condition=complete job/kube-bench-master --timeout=180s
  3. Read the summary and the failures.

    Terminal window
    kubectl logs job/kube-bench-master | grep -E '^\[FAIL\]' | head -15
    kubectl logs job/kube-bench-master | grep -A6 '== Summary'
  4. Each failure comes with a remediation. Read one and find the setting it refers to.

    Terminal window
    kubectl logs job/kube-bench-master | grep -A4 -E '^1\.2\.[0-9]+ ' | head -20
    docker exec kind-control-plane grep -E -- '--(profiling|audit-log-path)' /etc/kubernetes/manifests/kube-apiserver.yaml || echo "flag not set"

What you should see: A list of PASS, FAIL, and WARN results by benchmark section, with a summary at the end. On a default kind cluster expect failures such as no audit logging and profiling left on. Each remediation names the flag and the manifest to edit. Not every failure applies to your environment; record the ones you accept and why.

Note: The kube-bench image comes from Docker Hub, not registry.k8s.io.

Clean up:

Terminal window
rm -f job-master.yaml
kubectl config set-context --current --namespace=default
kubectl delete namespace cks-cis

Practise

  • Run kube-bench on a control plane node and fix two failing checks.

Know

  • Reference a kubernetes.io/tls Secret in the Ingress tls section; the host must match the certificate.

Read

Demo: Serve an Ingress over HTTPS with your own certificate

  1. Create a namespace for this demo and make it the default for your context.

    Terminal window
    kubectl create namespace cks-tls
    kubectl config set-context --current --namespace=cks-tls
  2. Create a certificate for the host and store it as a TLS Secret.

    Terminal window
    openssl req -x509 -newkey rsa:2048 -nodes -days 1 -keyout tls.key -out tls.crt -subj '/CN=secure.example' -addext 'subjectAltName=DNS:secure.example'
    kubectl create secret tls secure-tls --cert=tls.crt --key=tls.key
  3. An Ingress that uses it.

    Terminal window
    kubectl create deployment web --image=registry.k8s.io/e2e-test-images/agnhost:2.66.1 -- /agnhost netexec --http-port=8080
    kubectl expose deployment web --port=80 --target-port=8080
    kubectl apply -f - <<'EOF'
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
    name: secure
    spec:
    ingressClassName: cloud-provider-kind
    tls:
    - hosts: ["secure.example"]
    secretName: secure-tls
    rules:
    - host: secure.example
    http:
    paths:
    - path: /
    pathType: Prefix
    backend:
    service:
    name: web
    port:
    number: 80
    EOF
    kubectl get ingress secure -w # wait for an ADDRESS, then Ctrl-C
  4. Connect with the right name and check what certificate is served.

    Terminal window
    IP=$(kubectl get ingress secure -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    docker cp tls.crt kind-control-plane:/tmp/secure.crt
    docker exec kind-control-plane curl -s --cacert /tmp/secure.crt --resolve secure.example:443:$IP https://secure.example/hostname; echo
    docker exec kind-control-plane curl -sv --cacert /tmp/secure.crt --resolve secure.example:443:$IP https://secure.example/ 2>&1 | grep -E 'subject:|SSL connection'

What you should see: curl verifies the certificate against your CA file and returns a pod name. The verbose output shows subject: CN=secure.example. If the host in tls.hosts doesn’t match the certificate’s names, clients fail verification. In production, cert-manager issues and renews these Secrets for you.

Clean up:

Terminal window
rm -f tls.key tls.crt
kubectl config set-context --current --namespace=default
kubectl delete namespace cks-tls

Practise

  • Create a self-signed certificate, store it as a TLS Secret, and serve an Ingress over HTTPS.

Know

  • Block pod access to cloud metadata endpoints and to the kubelet API with NetworkPolicies and kubelet authentication settings.

Read

Demo: Lock down the kubelet and keep pods away from it

  1. Create a namespace for this demo and make it the default for your context.

    Terminal window
    kubectl create namespace cks-nodeprotect
    kubectl config set-context --current --namespace=cks-nodeprotect
  2. Confirm the kubelet’s API needs authentication and its read-only port is off.

    Terminal window
    kubectl get --raw /api/v1/nodes/kind-worker/proxy/configz | jq '.kubeletconfig | {anonymous: .authentication.anonymous.enabled, webhookAuthn: .authentication.webhook.enabled, authz: .authorization.mode, readOnlyPort}'
    docker exec kind-control-plane curl -s -o /dev/null -w '%{http_code}\n' http://kind-worker:10255/pods || true
  3. Block pods from reaching node addresses at all.

    Terminal window
    kubectl get nodes -o jsonpath='{range .items[*]}{.status.addresses[0].address}{"\n"}{end}'
    kubectl apply -f - <<'EOF'
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
    name: no-node-access
    spec:
    podSelector: {}
    policyTypes: ["Egress"]
    egress:
    - to:
    - ipBlock:
    cidr: 0.0.0.0/0
    except: ["172.18.0.0/16", "169.254.169.254/32"]
    EOF
    kubectl run probe --image=registry.k8s.io/e2e-test-images/busybox:1.37.0-2 -- sleep 3600 && kubectl wait --for=condition=Ready pod/probe --timeout=60s
    kubectl exec probe -- sh -c 'nc -zv -w 2 kind-worker 10250' || echo blocked

What you should see: anonymous is false, authorization is Webhook, and the read-only port 10255 doesn’t answer. After the policy the probe can’t reach the kubelet port. The node addresses on kind usually sit in 172.18.0.0/16 (the Docker kind network); check the printed IPs and adjust the CIDR to match your nodes.

Clean up:

Terminal window
kubectl config set-context --current --namespace=default
kubectl delete namespace cks-nodeprotect

Know

  • Compare binary checksums or signatures against the published release artifacts before installing.

Read

Demo: Check kubectl by checksum and by signature

  1. Download a release binary with its checksum and signature files, and check the checksum.

    Terminal window
    URL=https://dl.k8s.io/release/v1.37.2/bin/linux/amd64
    for f in kubectl kubectl.sha256 kubectl.sig kubectl.cert; do curl -sSfL --retry 3 "$URL/$f" -o "$f"; done
    echo "$(cat kubectl.sha256) kubectl" | sha256sum --check
  2. Verify the signature with cosign (v3.1.4), using the identity the Kubernetes release process signs with.

    Terminal window
    cosign verify-blob kubectl \
    --signature kubectl.sig \
    --certificate kubectl.cert \
    --certificate-identity krel-staging@k8s-releng-prod.iam.gserviceaccount.com \
    --certificate-oidc-issuer https://accounts.google.com
  3. Tamper with the binary and check again.

    Terminal window
    echo x >> kubectl
    echo "$(cat kubectl.sha256) kubectl" | sha256sum --check

What you should see: The checksum line says kubectl: OK and cosign prints Verified OK (cosign v3 also warns that --signature and --certificate are deprecated in favour of bundles; the commands are the ones in the official Kubernetes docs). After tampering the checksum fails. A checksum from the same server only proves the download wasn’t corrupted; the signature proves who built it.

Clean up:

Terminal window
rm -f kubectl kubectl.sha256 kubectl.sig kubectl.cert

Practise

  • Verify a downloaded kubectl against its published SHA-256 checksum.

Domain and competency names are from the CNCF CKS curriculum (v1.37), cncf/curriculum, licensed CC BY 4.0. Summaries, demos, and exercises are original to this site.