Node NotReady
The node’s Ready condition is False or Unknown. False means the kubelet is reporting a problem. Unknown means the control plane hasn’t heard from the kubelet at all.
After a timeout, the node lifecycle controller taints the node (node.kubernetes.io/unreachable or not-ready). By default, pods tolerate those taints for 300 seconds, then they’re evicted and replaced elsewhere.
Diagnose in order
Section titled “Diagnose in order”- Read the condition message:
kubectl describe node worker-2 | sed -n '/Conditions:/,/Addresses:/p'. - If the status is
Unknown, the kubelet isn’t reaching the API server: the node is down, the kubelet crashed, or the network path is broken. - On the node (SSH, or
kubectl debug node/worker-2 -it --image=busybox:1.36if it’s still partly reachable):systemctl status kubeletandjournalctl -u kubelet -n 100 --no-pager. Check the container runtime too:systemctl status containerdorcrio.
Usual causes
Section titled “Usual causes”| Message or log | Cause |
|---|---|
container runtime network not ready ... cni plugin not initialized |
CNI pods not running on the node |
PLEG is not healthy |
The container runtime is unresponsive or overloaded |
MemoryPressure / DiskPressure true before going NotReady |
The node starved its own components; see Evicted |
| Kubelet won’t start, cgroup driver mismatch | Kubelet and runtime configured differently, common after upgrades |
x509: certificate has expired in kubelet logs |
Kubelet client certificate not rotated; see Certificate has expired |
Fix what the kubelet log names. While you investigate, cordon the node so nothing new lands on it. If the node is gone for good, delete the Node object so its pods are cleaned up promptly.
Read next: Node pressure and eviction.