Multus PODs are stuck in ContainerCreating state due to "error getting pod: Unauthorized"

Issue

• Multus daemon sets are stuck in the ContainerCreating state and throwing the below errors:

  4m7s        Warning   FailedCreatePodSandBox   pod/multus-bhltv                                  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_multus-bhltv_openshift-multus_xyz_0(Container_ID): error adding pod openshift-multus_multus-bhltv to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): Multus: [openshift-multus/multus-bhltv/Container_ID]: error getting pod: Unauthorized
  

• The other PODs in the cluster are also not running because Multus PODs are not running on the nodes.