Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

Chapter 3. Locate an Idle Node

To backup a running cluster, identify an idle node as the back up target by connecting to the cluster via the HAProxy server; then, query the database to identify the host it is running on. The resulting node is the active node. Select one of the other hot standby nodes as the backup target.

For the purposes of a backing up a high availability cluster, Red Hat assumes at least three Galera cluster nodes. For example, overcloud nodes overcloud-controller-0, overcloud-controller-1 and overcloud-controller-2: 

overcloud-controller-0.example.com 192.168.1.1
overcloud-controller-1.example.com 192.168.1.2
overcloud-controller-2.example.com 192.168.1.3

Procedure

  1. Retrieve the virtual IP address HAProxy is listening on to identify which node is active. 

    [root@overcloud-controller-0]# grep -A1 mysql /etc/haproxy/haproxy.cfg

    The result will look similar to this: 

    listen mysql
  bind 192.0.2.18:3306

  2. Retrieve the username and password of a user allowed to connect to the virtual IP address. For example, the nova user and password from /etc/nova/nova.conf typically has these credentials. 

    [root@overcloud-controller-0 heat-admin]# crudini --get /etc/nova/nova.conf database connection

    Or 

    [root@overcloud-controller-0 heat-admin]# grep mysql /etc/nova/nova.conf

    The result should look something like this: 

    connection=mysql+pymysql://nova:xvsZqeaJn2fYwMK8NbscAJ6xG@172.16.2.5/nova

    The username and password in the foregoing example are nova and xvsZqeaJn2fYwMK8NbscAJ6xG respectively. The next step requires the username and password retrieved in this step.

  3. Connect to the database over the virtual IP address to get the name of the targeted node. 

    [root@overcloud-controller-0]# mysql -u <username> -p -h 192.0.2.18 -nNE -e "show variables like 'hostname';"
Enter password: ****

    Replace <username> with a user allowed to connect to the virtual IP address. The command line interface will prompt for a password. The result should look similar to this: 

    ************************ 1. row ************************
hostname
overcloud-controller-0.example.com

    In the foregoing example, overcloud-controller-0 is the active node targeted by HAProxy.

  4. Ensure the remaining hot standby nodes are connected and in sync. In the following example, assume overcloud-controller-1 and overcloud-controller-2 are the hot standby nodes. 

    [root@overcloud-controller-0]# curl overcloud-controller-1.example.com:9200 overcloud-controller-2.example.com:9200

    If the remaining nodes are connected and in sync with the active node targeted by HAProxy, the result should look similar to this: 

    Galera cluster node is synced.
Galera cluster node is synced.
    Important

    Execute the backup procedure on a hot standby node that is connected to the cluster and synchronized with the active node that HAProxy is targeting.

In the foregoing example, both overcloud-controller-1 and overcloud-controller-2 are connected and synchronized. Identify a connected and synchronized hot standby node to backup the overcloud.