Submariner connectivity unhealthy after IBM ROKS cluster upgrade due to stale node labeling

Solution Verified - Updated -

Issue

  • ACM Submariner Addon shows Unhealthy After IBM ROKS Cluster Upgraded from 4.20 to 4.21.

  • After upgrading an IBM ROKS managed cluster, the Submariner connectivity becomes unhealthy. The Submariner addon controller on the ACM Hub fails to automatically label the new worker nodes as gateways.

    • The ManagedClusterAddOn for submariner on the Hub shows Unhealthy or Degraded.
    • In the managed cluster, no nodes are labeled with submariner.io/gateway.
    • Tunnels are down between clusters.
    • The SubmarinerConfig status on the Hub references node names that no longer exist in the managed cluster.

Environment

  • Red Hat Advanced Cluster Management for Kubernetes 2.16
  • Submariner Addon
  • Managed clusters on IBM Cloud (ROKS)
  • ROKS cluster upgrade (e.g., 4.20 to 4.21) involving worker node replacement

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content