ODF upgrade from 4.18 to 4.19 stuck with NooBaa in "Creating" phase and "dbClusterStatus" stuck in "Importing"

Solution Verified - Updated -

Issue

The ODF 4.19 upgrade migrates the NooBaa database from the existing single PostgreSQL instance to a highly available PostgreSQL cluster managed by CloudNativePG (CNPG). If you are hitting this issue, the following is observed:

  • ODF upgrade completed successfully; however, the StorageCluster CR remains in phase Progressing due to the NooBaa CR failing to reconcile with a message: Waiting on Nooba instance to finish initialization.

  • NooBaa S3 services remained unavailable until manual intervention was taken.

  • The NooBaa CR remains in phase Creating with NooBaa CR "message: cluster creation or import is still in progress, skipping update," and the dbClusterStatus: Importing indefinitely.

  • An s3 outage remains as the noobaa-core and noobaa-endpoint pods fail to scale up due to the upgrade sequence requiring the dbClusterStatus to be Ready, ** WARNING: DO NOT MANUALLY SCALE UP noobaa-core and noobaa-endpoint.**

  • The CNPG Cluster reports Phase: Cluster in healthy state

  • The noobaa-operator logs contain the following warnings and errors repeatedly:

  • BackingStore/NamespaceStore remain in phase Connecting

    level=info msg="cnpg:: the cluster spec was changed but the cluster creation or import is still in progress, skipping update" sys=openshift-storage/noobaa
    level=error msg="cnpg:: got error reconciling cluster. error: cluster creation or import is still in progress, skipping update" sys=openshift-storage/noobaa
    level=warning msg="⏳ Temporary Error: cluster creation or import is still in progress, skipping update" sys=openshift-storage/noobaa
    
  • The noobaa-db-pg-0 pod remains alongside noobaa-db-pg-cluster-1 and noobaa-db-pg-cluster-2 pods.

Environment

  • Red Hat OpenShift Container Platform (RHOCP) 4.x
  • Red Hat OpenShift Data Foundation (ODF) 4.19.x (upgrading from 4.18.x)
  • Multi-Cloud Object Gateway (MCG) / NooBaa

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content