Recycling of persistent volumes fails

Issue

• Having an issue during the recycling process of the persistent volumes:

  E0122 11:44:50.840753   22676 persistentvolume_recycler_controller.go:143] PersistentVolume[lv-1g-004] failed recycling: Pod failed, pod.Status.Message unknown.
  

• Why does the recycling of persistent volumes fail?
• When a pod is deleted and the pv has to be freed up, there is a recycler pod that comes in and tries to make the PV available. But, every mount has a .snapshot folder that is readonly as given to us by the storage team. Since the recycler pod fails to remove it, the recycler pod itself goes to failed state and the PV never goes to available. The PV then also stays in Failed state. To clean this up, I have to remove the recycler pod, remove the PV and recreate the PV.