Resources get stuck in a terminating state

Solution Verified - Updated -

Issue

Resources stuck in terminating state

  • Pods
  • Namespaces
  • Persistent Volumes

Pods

Pods are stuck in a 'Terminating' state when trying to delete a batch of pods. Restarting crio and kubelet will not delete the pods.

Namespaces

Namespaces get stuck in a 'Terminating' state when trying to delete a batch of namespaces. Force deleting pods in said namespace will help delete these namespaces.

Persistent Volumes

If for some reason you need to force-delete a pod, it may happen that the volume remains mounted in the node. Then diskmaker cannot clean it and it will fail with messages like:

Deleting PV block volume "local-pv-96cf56b9" device hostpath "/mnt/local-storage/general/dm-name-autopart-lv_9", mountpath "/mnt/local-storage/general/dm-name-autopart-lv_9"
Cleanup pv "local-pv-96cf56b9": StdoutBuf - "Calling mkfs"
Cleanup pv "local-pv-96cf56b9": StdoutBuf - "mke2fs 1.45.6 (20-Mar-2020)"
Cleanup pv "local-pv-96cf56b9": StdoutBuf - "/mnt/local-storage/general/dm-name-autopart-lv_9 is apparently in use by the system; will not make a filesystem here"

Environment

Openshift Container Platform 4.8.28 and earlier
Openshift Container Platform 4.9.17 and earlier
Openshift Container Platform 4.10.2 and earlier

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content