Heketi fails to remove node and its devices in an event of deprecating a glusterfs node on Openshift

Solution Verified - Updated -

Issue

Deprecate a node where a glusterfs node is working on and scale a new node on Openshift and Heketi topology:

  1. Node/VM dies or has to be replaced for maintenance purposes;
  2. Deprecate node in Openshift by removing it from the cluster;
  3. Scale the Glusterfs pods to 2 taking in consideration both points above;
  4. Reconfigure heketi db to reflect the new topology;
  5. Add or activate the node in Openshift cluster;
  6. Probe the node back to the Glusterfs Trusted Storage Pool;
  7. Add the node back to the heketi topology with the respective storage device where the bricks will be created.

Without adding the replacement node, heketi will give errors when attempting to remove the old/failed node and its devices:

Error: Failed to remove device, error: Id not found

Environment

Openshift Container Plataform and Openshift Container Storage v3.9, v3.10 and v3.11
This has only been tested with a standard deployment of 3 Glusterfs containerized nodes.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In