How to Automatically configure a 2 node Pacemaker cluster for managing HANA Scale-Up System Replication using Ansible?

Updated -

Contents

1. Overview

This article describes how to automate the setup of a 2 node Pacemaker cluster for managing HANA Scale-Up System Replication using Ansible. The goal is to have a single ansible-playbook call that will take care of configuring the entire environment, starting from configuring the cluster nodes for running SAP HANA, installing SAP HANA on the cluster nodes, configuring SAP HANA System Replication and then configuring the cluster to manage the SAP HANA Scale-Up System Replication environment.

The cluster setup that will be created by these ansible playbooks is based on the support policies and configuration guides mentioned at Supported HA Scenarios for SAP HANA, SAP S/4HANA, and SAP NetWeaver

Note: This solution is Technology Preview. Red Hat Global Support Services may create bug reports on behalf of support cases filed by subscribed customers.

For more information on how to configure ansible and use system roles please refer to Red Hat Enterprise Linux System Roles for SAP.

The main steps are outlined as follows:

  • Prepare ansible control node and managed nodes
  • Download the ansible collections and the installation media
  • Create the yaml files for the variable definitions and the playbook
  • Run the playbook to create the complete environment

1.1 Preparation

  • Base installation of cluster nodes
    • Ensure that there are no conflicting entries in the /etc/hosts file
  • Ansible Master Node is available
    • ansible >=2.9 is installed
  • SSH access for the user who runs the ansible playbook to the managed cluster nodes. Please also check the documentation.
  • Sudo entry for the user on managed nodes
    • Unrestricted privileges required
    • Sudo entry for the root user required if root user is used for running the playbook (unless any other become-method is explicitly configured in ansible)
  • RHEL for SAP Solutions subscriptions for all cluster nodes
  • SAP HANA installation files stored in /software/hana (configurable) per managed node.

1.2 Out of Scope

This collection does not cover
- firewall settings
- SAP HANA software license
- SAP HANA Scale-Out Cluster Implementation.

2. Simple example

  • Configure inventory
  • Accessing system roles collection
  • Edit playbook with parameters
  • Add HANA software
  • Run playbook

2.1 Configuring Inventory

The default inventory file is /etc/ansible/hosts. Please add a group name and a list of nodes into the inventory file.

Example:

[cluster1]
hana01
hana02

2.2 Accessing system roles collection

The system roles for SAP Solutions can be downloaded from github or as a package:

If they are extracted either using git or extracting a package you will find a directory named roles.
You can, for example, create a link to these roles in the directory where your playbook is stored. Ansible should see either the roles directory or set the roles_path in the ansible.cfg file.

The roles directory looks like this for example:

community.sap_install/roles]$ ll | awk '{ print $NF }'
0
sap_general_preconfigure
sap_ha_install_hana_hsr
sap_ha_install_pacemaker
sap_hana_install
sap_hana_preconfigure
sap_ha_prepare_pacemaker
sap_ha_set_hana
sap_ha_set_netweaver
sap_hostagent
sap_install_media_detect
sap_netweaver_preconfigure
sap_storage
sap_swpm

2.3 Playbook file

In this simple example, roles and variables can be combined into a single file. Here is an example of a single playbook file.

more simple_one_file_example.yml

---
# call playbook with # ansible-playbook simple_one_file_example.yml
- name: "Example SAP Hana and HA Cluster deployment on a 2-node cluster"
  hosts: hana1, hana2
  become: true

  vars:
    sap_domain: example.com

    sap_hana_sid: 'DB1'
    sap_hana_instance_number: '00'
    sap_hana_install_master_password: 'my_hana_password'
    # It is recommended to use vault instead of plain text passwords. Example please check README

    ### Cluster Definition
    sap_ha_install_pacemaker_cluster_name: cluster1
    sap_hana_hacluster_password: 'my_hacluster'

    sap_hana_cluster_nodes:
      - node_name: hana1
        node_ip: 10.0.0.101
        node_role: primary
        hana_site: DC01

      - node_name: hana2
        node_ip: 10.0.0.102
        node_role: secondary
        hana_site: DC02

    sap_ha_set_hana_vip1: 10.0.0.200

    sap_pacemaker_stonith_devices:
      - name: "fence_name_for_rhevm"
        agent: "fence_rhevm"
        parameters: "disable_http_filter=1 ipaddr=rhev-manager.example.com login='rhevuser@internal' password=rhevuser-pw pcmk_host_map='hana1:hana1;hana2:hana2' power_wait=3 ssl=1 ssl_insecure=1"

  roles:

    - sap_general_preconfigure
    - sap_hana_preconfigure
    - sap_hana_install
    - sap_ha_install_hana_hsr
    - sap_ha_prepare_pacemaker
    - sap_ha_install_pacemaker
    - sap_ha_set_hana

This stonith device example is covering rhevm. You will find further examples later.

2.4 HANA Software

The roles are expecting the HANA installation software specified by the variable:

sap_hana_install_software_directory: /software/hana

Having the installation media into this directory is recommended, otherwise you have to adapt the variable.
For more information please check README of the sap_hana_install role

2.5 Execute Playbook

The last step is to call the playbook:

ansible-playbook simple_one_file_example.yml

Another option is to call the playbook with different options to get additional information. This can be necessary for troubleshooting or for higher transparency.

If you want to check and see what the playbook is doing you can use:

ansible-playbook simple_one_file_example.yml --list-tasks

This will list all the tasks without performing them.
Another option is the verbose option. You can add one or more v. This will apply changes, but will give you more details. Example:

ansible-playbook simple_one_file_example.yml -vvv

2.6 Optional Execution

In the example above everything is part of a single playbook. If you want to configure multiple environments similar.
If you create another parameter file for example:

ansible-playbook simple_one_file_example.yml -e @parameterfile_with_new_environment.yml

The parameterfile_with_new_environment.yml might look like:

sap_ha_install_pacemaker_cluster_name: cluster2
sap_hana_hacluster_password: 'my_hacluster2'

sap_hana_cluster_nodes:
  - node_name: hana3
    node_ip: 10.0.0.103
    node_role: primary
    hana_site: DC03

  - node_name: hana4
    node_ip: 10.0.0.104
    node_role: secondary
    hana_site: DC04

sap_ha_set_hana_vip1: 10.0.0.202

sap_pacemaker_stonith_devices:
  - name: "fence_name_for_rhevm"
    agent: "fence_rhevm"
    parameters: "disable_http_filter=1 ipaddr=rhev-manager.example.com login='rhevuser@internal' password=rhevuser-pw pcmk_host_map='hana3:hana3;hana4:hana4' power_wait=3 ssl=1 ssl_insecure=1"

3. Fencing

Below are some examples for setting up the fence devices for various platforms using Ansible. Please also check the official documentation. In most cases it will be required to change the IP-Address, User and Password as per your environment

As documented in Support Policies for RHEL High Availability Clusters - General Requirements for Fencing/STONITH having fencing/STONITH enabled and working is a requirement for having a supported cluster environment.

NOTE: It is required to physically verify the fencing configuration before putting the cluster in production. You can use the following link for guidance:
How to test fence devices and fencing configuration in a Red Hat High Availability cluster?

3.1 AWS

sap_pacemaker_stonith_devices:
  - name: "fence_name_for_aws"
    agent: "fence_aws"
    parameters: "secret_key=secret-access-key region=region pcmk_host_map="rhel-hostname-1:Instance-ID-1;rhel-hostname-2:Instance-ID-2;rhel-hostname-3:Instance-ID-3" power_timeout=240 pcmk_reboot_timeout=480 pcmk_reboot_retries=4"

3.2 RHEVM

sap_pacemaker_stonith_devices:
  - name: "fence_name_for_rhevm"
    agent: "fence_rhevm"
    parameters: "disable_http_filter=1 ipaddr=rhev-manager.example.com login='rhevuser@internal' password=rhevuser-pw pcmk_host_map='hana3:hana3;hana4:hana4' power_wait=3 ssl=1 ssl_insecure=1"

3.3 VMWARE

sap_pacemaker_stonith_devices:
  - name: "fence_name_for_vmware"
    agent: "fence_vmware_soap"
    parameters: "ipaddr=esxi_server.example.com login='esxi-user' password=esxi-pw pcmk_host_map='hana3:hana3;hana4:hana4' power_wait=3 ssl=1 ssl_insecure=1"

3.4 HMC

sap_pacemaker_stonith_devices:
  - name: "fence_name_for_lpar"
    agent: "fence_lpar"
    parameters: "ipaddr=hmc.example.com login='hmc-user' password=hmc-pw pcmk_host_map='hana3:hana3;hana4:hana4'"

3.5 AZURE

Please check the AZURE Online Documentation

4. References

Comments