Aller au contenu

KubePodNotReady#

It means that the pod is non-ready. Lot of triggers can cause this alert.

Troobleshooting hints#

  • Check the pod logs
  • kubectl describe pod and look at the events.

Known Case - problem with mount of pvc in pod#

This procedure can be applied when we have the following output to a pod describe command: 

 MountVolume.WaitForAttach failed for volume "pvc-xxxxx" : waitforattach command failed, status: Failed.

Resolution without losing data#

1/ Patch pvc while keeping the associated pv

kubectl patch pv/pvc-<pvc_id> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

Note

Useful only if the policy is not Retain.

2/ Delete the pod (that is using <pvc_id>) and pvc itself

Example with prometheus cloud-client:

kubectl -n monitoring-client delete prometheus client-prometheus
kubectl -n monitoring-client delete pvc prometheus-client-prometheus-db-prometheus-client-prometheus-1

3/ Update the status of pv

In order to change its status from Released to Available:

kubectl patch pv/pvc-<pvc_id> --type json -p='[{"op": "remove", "path": "/spec/claimRef"}]'

4/ Redeploy pod and pvc

Example with prometheus cloud-client:

trackbone plan -z <cloud_zone_name> -c kube-prometheus-stack-cloud-client
trackbone apply -z <cloud_zone_name> -c kube-prometheus-stack-cloud-client

5/ Patch policy pv to return to the init state 

kubectl patch pv/pvc-<pvc_id> -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}'

Note

Useful only if the policy is not Retain.