Chapter 2. Installing and configuring Instance HA

You use Red Hat OpenStack Platform (RHOSP) director to deploy Instance High Availability (HA). However, you must perform additional steps to configure a new Instance HA deployment on a new overcloud. After you complete the steps, Instance HA will run on a subset of Compute nodes with a custom role.

Important

Instance HA is not supported on RHOSP hyperconverged infrastructures (HCI) environments. To use Instance HA in your RHOSP HCI environment, you must designate a subset of the Compute nodes with the ComputeInstanceHA role to use the Instance HA. Red Hat Ceph Storage services must not be hosted on the Compute nodes that host Instance HA.

Important

To enable instance HA in a different environment, such as an existing overcloud that uses standard or custom roles, perform only the procedures that are relevant to your deployment and adapt your templates accordingly.

2.1. Configuring the Instance HA role and profile

Before you deploy Instance HA, add the Instance HA role to your roles-data.yaml file, tag each Compute node that you want to manage with Instance HA with the Instance HA profile, and add these to your overcloud-baremetal-deploy.yaml file or equivalent. For more information about designating overcloud nodes for specific roles, see: Designating overcloud nodes for roles by matching profiles. As an example, you can use the computeiha profile to configure the node.

Procedure

  1. Check the existing capabilities of each registered node: 

    (undercloud)$ openstack baremetal node show <node> -f json -c properties | jq -r .properties.capabilities

  2. Assign the profile capability to each bare metal node that you want to match to a role profile, by adding profile:computeiha to the existing capabilities of the node: 

    (undercloud)$ openstack baremetal node set  <node> --property capabilities="profile:computeiha,<capability_1>,...,<capability_n>"
    • Replace <node> with the ID of the bare metal node.
    • Replace <capability_1>, and all capabilities up to <capability_n>, with each capability that you checked in step 1.
  3. Add the role to your overcloud-baremetal-deploy.yaml file, if not already defined.

  4. Edit overcloud-baremetal-deploy.yaml to define the profile that you want to assign to the nodes for the role: 

    - name: ComputeInstanceHA
  count: 2
  hostname_format: compute-%index%
  defaults:
    profile: computeiha
    network_config:
      template: /home/stack/composable_roles/network/nic-configs/compute.j2
    networks:
    - network: ctlplane
      vif: true
    - network: internal_api
    - network: tenant
    - network: storage

  5. Provision the overcloud nodes: 

    (undercloud)$ openstack overcloud node provision \
--stack <stack> \
--output <deployment_file> \
/home/stack/templates/overcloud-baremetal-deploy.yaml
    • Replace <stack> with the name of the stack for which you provisioned the bare-metal nodes. The default value is overcloud.
    • Replace <deployment_file> with a name that you choose for the generated heat environment file to include with the deployment command, for example /home/stack/templates/overcloud-baremetal-deployed.yaml.

2.2. Enabling fencing on an overcloud with Instance HA

Enable fencing on all Controller and Compute nodes in the overcloud by creating an environment file with fencing information.

Procedure

  1. Create the environment file in an accessible location, such as ~/templates, and include the following content: 

    parameter_defaults:
  EnableFencing: true
  FencingConfig:
    devices:
    - agent: fence_ipmilan
      host_mac: 00:ec:ad:cb:3c:c7
      params:
        login: admin
        ipaddr: 192.168.24.1
        ipport: 6230
        passwd: password
        lanplus: 1
    - agent: fence_ipmilan
      host_mac: 00:ec:ad:cb:3c:cb
      params:
        login: admin
        ipaddr: 192.168.24.1
        ipport: 6231
        passwd: password
        lanplus: 1
    - agent: fence_ipmilan
      host_mac: 00:ec:ad:cb:3c:cf
      params:
        login: admin
        ipaddr: 192.168.24.1
        ipport: 6232
        passwd: password
        lanplus: 1
    - agent: fence_ipmilan
      host_mac: 00:ec:ad:cb:3c:d3
      params:
        login: admin
        ipaddr: 192.168.24.1
        ipport: 6233
        passwd: password
        lanplus: 1
    - agent: fence_ipmilan
      host_mac: 00:ec:ad:cb:3c:d7
      params:
        login: admin
        ipaddr: 192.168.24.1
        ipport: 6234
        passwd: password
        lanplus: 1

  2. If you do not use shared storage for your Compute instance, add the following parameter to the environment file that you created: 

    parameter_defaults:
  ExtraConfig:
    tripleo::instanceha::no_shared_storage: true

Additional resources

2.3. Deploying the overcloud with Instance HA

If you already deployed the overcloud, you can run the openstack overcloud deploy command again with the additional Instance HA files you created. You can configure Instance HA for your overcloud at any time after you create the undercloud.

Prerequisites

  • You configured a Instance HA role and profile.
  • You enabled fencing on the overcloud.

Procedure

  • Use the openstack overcloud deploy command with the -e option to include the compute-instanceha.yaml environment file and to include additional environment files. 

    $ openstack overcloud deploy --templates \
 -e <fencing_environment_file> \
 -r my_roles_data.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/compute-instanceha.yaml

    Replace <fencing_environment_file> with the appropriate file names for your environment:

Note
  • Do not modify the compute-instanceha.yaml environment file.
  • Include the full path to each environment file that you want to include in the overcloud deployment.

After deployment, each Compute node includes a STONITH device and a pacemaker_remote service.

2.4. Testing Instance HA evacuation

To test that Instance HA evacuates instances correctly, you trigger evacuation on a Compute node and check that the Instance HA agents successfully evacuate and re-create the instance on a different Compute node.

Warning

The following procedure involves deliberately crashing a Compute node, which triggers the automated evacuation of instances with Instance HA.

Prerequisites

  • Instance HA is deployed on the Compute node.

Procedure

  1. Start one or more instances on the overcloud. 

    stack@director $ . overcloudrc
stack@director $ openstack server create --image cirros --flavor 2 test-failover
stack@director $ openstack server list -c Name -c Status

  2. Log in to the Compute node that hosts the instances and change to the root user. Replace compute-n with the name of the Compute node: 

    stack@director $ . stackrc
stack@director $ ssh -l tripleo-admin compute-n
tripleo-admin@compute-n $ su -

  3. Crash the Compute node. 

    root@compute-n $ echo c > /proc/sysrq-trigger

  4. Wait a few minutes for the node to restart, and then verify that the instances from the Compute node that you crash are re-created on another Compute node: 

    stack@director $ openstack server list -c Name -c Status
stack@director $ openstack compute service list

2.5. Designating instances to evacuate with Instance HA

By default, Instance HA evacuates all instances from a failed node. You can configure Instance HA to only evacuate instances with specific images or flavors.

Prerequisites

  • Instance HA is deployed on the overcloud.

Procedure

  1. Log in to the undercloud as the stack user.

  2. Source the overcloudrc file: 

    $ source ~/overcloudrc

  3. Use one of the following options:

    • Tag an image: 

      (overcloud) $ openstack image set --tag evacuable <image_id>

      Replace <image_id> with the ID of the image that you want to evacuate.

    • Tag a flavor: 

      (overcloud) $ openstack flavor set --property evacuable=true <flavor_id>

      Replace <flavor_id> with the ID of the flavor that you want to evacuate.

2.6. Additional resources