Chapter 7. Troubleshooting and deleting remaining resources during Uninstall

Occasionally some of the custom resources managed by an operator may remain in "Terminating" status waiting on the finalizer to complete, although you have performed all the required cleanup tasks. In such an event you need to force the removal of such resources. If you do not do so, the resources remain in the "Terminating" state even after you have performed all the uninstall steps.

  1. Check if the openshift-storage namespace is stuck in Terminating state upon deletion.

    $ oc get project openshift-storage

    Output:

    NAME                DISPLAY NAME   STATUS
    openshift-storage                  Terminating
  2. Check for the NamespaceFinalizersRemaining and NamespaceContentRemaining messages in the STATUS section of the command output and perform the next step for each of the listed resources.

    $ oc get project openshift-storage -o yaml

    Example output :

    status:
      conditions:
      - lastTransitionTime: "2020-07-26T12:32:56Z"
        message: All resources successfully discovered
        reason: ResourcesDiscovered
        status: "False"
        type: NamespaceDeletionDiscoveryFailure
      - lastTransitionTime: "2020-07-26T12:32:56Z"
        message: All legacy kube types successfully parsed
        reason: ParsedGroupVersions
        status: "False"
        type: NamespaceDeletionGroupVersionParsingFailure
      - lastTransitionTime: "2020-07-26T12:32:56Z"
        message: All content successfully deleted, may be waiting on finalization
        reason: ContentDeleted
        status: "False"
        type: NamespaceDeletionContentFailure
      - lastTransitionTime: "2020-07-26T12:32:56Z"
        message: 'Some resources are remaining: cephobjectstoreusers.ceph.rook.io has
          1 resource instances'
        reason: SomeResourcesRemain
        status: "True"
        type: NamespaceContentRemaining
      - lastTransitionTime: "2020-07-26T12:32:56Z"
        message: 'Some content in the namespace has finalizers remaining: cephobjectstoreuser.ceph.rook.io
          in 1 resource instances'
        reason: SomeFinalizersRemain
        status: "True"
        type: NamespaceFinalizersRemaining
  3. Delete all the remaining resources listed in the previous step.

    For each of the resources to be deleted, do the following:

    1. Get the object kind of the resource which needs to be removed. See the message in the above output.

      Example :

      message: Some content in the namespace has finalizers remaining: cephobjectstoreuser.ceph.rook.io

      Here cephobjectstoreuser.ceph.rook.io is the object kind.

    2. Get the Object name corresponding to the object kind.

      $ oc get  <Object-kind> -n  <project-name>

      Example :

      $ oc get cephobjectstoreusers.ceph.rook.io -n openshift-storage

      Example output:

      NAME                           AGE
      noobaa-ceph-objectstore-user   26h
    3. Patch the resources.

      $ oc patch -n <project-name> <object-kind>/<object-name> --type=merge -p '{"metadata": {"finalizers":null}}'

      Example:

      $ oc patch -n openshift-storage cephobjectstoreusers.ceph.rook.io/noobaa-ceph-objectstore-user \
      --type=merge -p '{"metadata": {"finalizers":null}}'

      Output:

      cephobjectstoreuser.ceph.rook.io/noobaa-ceph-objectstore-user patched
  4. Verify that the openshift-storage project is deleted.

    $ oc get project  openshift-storage

    Output:

    Error from server (NotFound): namespaces "openshift-storage" not found

    If the issue persists, reach out to Red Hat Support.