How to handle Evicted pods in OpenShift

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP) 4.x

Issue

  • What is Evicted status pod?
  • Evicted pods won't be deleted automatically
  • What do we need to do with Evicted status pod?
  • How to clean Evicted pods?

Resolution

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

Pods become Evicted status as a result of eviction and you need to delete them manually.

Eviction happens when:

Kubernetes/OpenShift won't delete pods automatically other than normal scenario and try to keep them for record purposes. So these Evicted pods will not be deleted automatically.

In most cases, pod failover just works and you can delete Evicted pods manually. If you see some failures due to the eviction, the Evicted pod gives you some information like which node was failed.

As an OpenShift admin, you can list evicted pods by:

$ oc get pod --all-namespaces  | grep Evicted

and you can delete them all:

$ oc get pod --all-namespaces  | awk '{if ($4=="Evicted") print "oc delete pod " $2 " -n " $1;}' | sh
  • If you are using OpenShift Container Platform with v4.12 or newer, instead of Evicted search for ContainerStatusUnknown: oc get pod -A | grep ContainerStatusUnknown.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments