Storage cluster operator was reporting degraded status due to error: unable to find UUID for VM

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4
  • VMWare vSphere

Issue

  • During the cluster upgrade, Storage cluster operator was reporting degraded status due to "error: unable to find UUID for VM:".

Resolution

  • Validate the Datastore & Datacenter information in the 'cloud-config-provider' ConfigMap from 'openshift-config' Namespace. Make the necessary changes as required.
    [VirtualCenter "vcsa.example.com"]
    datacenter = "Datacenter01"   ----> Specify the appropriate Datacenter information
    datastore = "Datastore01"     ----> Specify the appropriate Datastore information

Root Cause

  • The VMs have been migrated from one Datacenter to another or from one Datastore to another.

Diagnostic Steps

  • Check providerID field on nodes, it shouldn't be present on the nodes.
$ oc get nodes <node_name> -oyaml | grep -i providerID
  • Check UUID on nodes.
$ oc get nodes <node_name> -oyaml | grep -i systemUUID
  • Apply the below taint on nodes where providerID is not present.
$ oc adm taint node <node_name> node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule

  • Even after applying the taint, the UUID doesn't get populated.

  • Please review the logs of vsphere-problem-detector-operator pods to identify where is it stuck.

$ oc get po -n openshift-cluster-storage-operator

$ oc logs -f po vsphere-problem-detector-operator<abc>

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments