Red Hat Training

A Red Hat training course is available for Red Hat JBoss Enterprise Application Platform

Chapter 7. Using JBoss EAP High Availability in Microsoft Azure

Microsoft Azure does not support JGroups discovery protocols that are based on UDP multicast. Although you may use other JGroups discovery protocols (such as a static configuration (TCPPING), a shared database (JDBC_PING), shared file system-based ping (FILE_PING), or TCPGOSSIP), we strongly recommend that you use the shared file discovery protocol specifically developed for Azure: AZURE_PING.

7.1. Configuring AZURE_PING for JBoss EAP High Availability

This section describes configuring your JBoss EAP cluster to use the AZURE_PING JGroups discovery protocol. Ensure that you meet the prerequisites when creating your virtual machines.

AZURE_PING uses a common blob container in a Microsoft Azure storage account. If you do not already have a blob container that AZURE_PING can use, create one that your virtual machines can access.

After creating your blob container, you will need the following information to configure AZURE_PING:

  • storage_account_name: the name of the Microsoft Azure storage account that contains your blob container.
  • storage_access_key: the secret access key of the storage account.
  • container: the name of the blob container to use for PING data.

To configure JBoss EAP to use AZURE_PING as the JGroups discovery protocol, you can either use a preconfigured example JBoss EAP configuration file, or modify an existing configuration.

Important

The following instructions configure AZURE_PING using a UDP JGroups stack. If you will be configuring JBoss EAP messaging high availability in Azure, you must configure AZURE_PING in a TCP JGroups stack instead.

7.1.1. Using the Example Configuration File

JBoss EAP includes example configuration files for configuring clustering of standalone servers in Microsoft Azure. These files are located in EAP_HOME/docs/examples/configs/ and are standalone-azure-ha.xml and standalone-azure-full-ha.xml.

Note

See the JBoss EAP Configuration Guide for an explanation of the differences between the server profiles.

These sample configuration files are preconfigured for using clustering in Microsoft Azure, and all that is needed is to specify the values for your Azure storage account and blob container.

Copy your desired example configuration file to EAP_HOME/standalone/configuration/.

7.1.2. Modifying an Existing Configuration

If you are modifying an existing JBoss EAP high availability configuration, the following changes to the jgroups subsystem are required.

  1. Launch the management CLI and embed a server to make offline changes to your chosen configuration file. For example: 

    $ EAP_HOME/bin/jboss-cli.sh
[disconnected /] embed-server --server-config=standalone-ha.xml

  2. By default, JGroups uses the UDP stack. If you were using another stack, change back to using the UDP stack: 

    [standalone@embedded /] /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=udp)

  3. Execute the following batch of commands to remove the existing UDP stack and insert a new UDP stack configured for Microsoft Azure: 

    batch
/subsystem=jgroups/stack=udp:remove
/subsystem=jgroups/stack=udp:add()
/subsystem=jgroups/stack=udp/transport=UDP:add(socket-binding=jgroups-udp,properties={ip_mcast=false})
/subsystem=jgroups/stack=udp/protocol=azure.AZURE_PING:add(properties={storage_account_name="${jboss.jgroups.azure_ping.storage_account_name}", storage_access_key="${jboss.jgroups.azure_ping.storage_access_key}", container="${jboss.jgroups.azure_ping.container}"})
/subsystem=jgroups/stack=udp/protocol=MERGE3:add
/subsystem=jgroups/stack=udp/protocol=FD_SOCK:add(socket-binding=jgroups-udp-fd)
/subsystem=jgroups/stack=udp/protocol=FD:add
/subsystem=jgroups/stack=udp/protocol=VERIFY_SUSPECT:add
/subsystem=jgroups/stack=udp/protocol=pbcast.NAKACK2:add(properties={use_mcast_xmit=false,use_mcast_xmit_req=false})
/subsystem=jgroups/stack=udp/protocol=UNICAST3:add
/subsystem=jgroups/stack=udp/protocol=pbcast.STABLE:add
/subsystem=jgroups/stack=udp/protocol=pbcast.GMS:add
/subsystem=jgroups/stack=udp/protocol=UFC:add
/subsystem=jgroups/stack=udp/protocol=FRAG2:add
run-batch
    Note

    If you want to store the values of your Microsoft Azure storage account and blob container in your configuration file, replace the system property references in the above configuration with the values from your Azure environment. In the following command, examples for starting JBoss EAP, the system properties are used.

    The stack XML in your configuration file should look like the following: 

    <stack name="udp">
    <transport type="UDP" socket-binding="jgroups-udp">
        <property name="ip_mcast">
            false
        </property>
    </transport>
    <protocol type="azure.AZURE_PING">
        <property name="storage_account_name">
            ${jboss.jgroups.azure_ping.storage_account_name}
        </property>
        <property name="storage_access_key">
            ${jboss.jgroups.azure_ping.storage_access_key}
        </property>
        <property name="container">
            ${jboss.jgroups.azure_ping.container}
        </property>
    </protocol>
    <protocol type="MERGE3"/>
    <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
    <protocol type="FD"/>
    <protocol type="VERIFY_SUSPECT"/>
    <protocol type="pbcast.NAKACK2">
        <property name="use_mcast_xmit">
            false
        </property>
        <property name="use_mcast_xmit_req">
            false
        </property>
    </protocol>
    <protocol type="UNICAST3"/>
    <protocol type="pbcast.STABLE"/>
    <protocol type="pbcast.GMS"/>
    <protocol type="UFC"/>
    <protocol type="FRAG2"/>
</stack>

  4. Stop the embedded server and exit the management CLI: 

    [standalone@embedded /] stop-embedded-server
[disconnected /] exit

7.2. Starting JBoss EAP High Availability

To start JBoss EAP using high availability in Microsoft Azure, you must:

  • use a configuration file that has been configured with the AZURE_PING discovery protocol and specify the required values of your Microsoft Azure storage account and blob container.

  • bind the private interface to the Microsoft Azure internal IP address that is used for clustering traffic. You can do this at startup, as shown below, or as a set configuration shown in the JBoss EAP Configuration Guide.

    Warning

    For security reasons, you must ensure that you do not expose clustering traffic to unintended networks.

    You can do this by restricting the endpoints to your Microsoft Azure virtual network or by creating a dedicated virtual network and dedicated virtual machine NICs for clustering traffic.

Start your JBoss EAP high availability instance using the following command. If you stored your Microsoft Azure storage account and blob container values in your configuration file, you can omit the -Djboss.jgroups.azure_ping system property definitions. 

EAP_HOME/bin/standalone.sh -b IP_ADDRESS -bprivate IP_ADDRESS --server-config=EAP_CONFIG_FILE.xml -Djboss.jgroups.azure_ping.storage_account_name=STORAGE_ACCOUNT_NAME -Djboss.jgroups.azure_ping.storage_access_key=STORAGE_ACCESS_KEY -Djboss.jgroups.azure_ping.container=CONTAINER_NAME

For example: 

EAP_HOME/bin/standalone.sh -b 172.28.0.2 -bprivate 172.28.0.2 --server-config=standalone-azure-ha.xml -Djboss.jgroups.azure_ping.storage_account_name=my_storage_account -Djboss.jgroups.azure_ping.storage_access_key=y7+2x7P68pQse9MNh58Bkk5po9OGzeJc+0IRqYcQ9Cr/Sp4xiUFJVlbY+MGXJRNx3syksikwm4tOYlFgjvoCmw== -Djboss.jgroups.azure_ping.container=my_blob_container
Note

As JBoss EAP subsystems only start when needed, you must deploy a distributable application to your JBoss EAP servers to start the high availability JBoss EAP subsystems.

After you start a second JBoss EAP instance in a cluster, you should see logs similar to the following in the console log of the first server in the cluster: 

INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (thread-2,ee,eap-server-1) ISPN000094: Received new cluster view for channel server: [eap-server-1|1] (2) [eap-server-1, eap-server-2]

7.3. Troubleshooting JBoss EAP High Availability

If you are having trouble getting clustering to work in Microsoft Azure, verify that you have completed all the requirements in the following list.

  • Ensure that the Microsoft Azure virtual machines hosting your JBoss EAP servers are using the same virtual network.
  • Ensure that you have a blob container for AZURE_PING to use.
  • Ensure that you are using a JBoss EAP configuration file with the AZURE_PING discovery protocol configured in the jgroups subsystem.
  • Ensure that you are binding both the public and private interfaces to the correct Microsoft Azure IP addresses.

  • Ensure that you have the correct values for your Microsoft Azure storage account and blob container and that you are either:

    • storing them in your configuration file, or
    • setting the correct system properties when starting JBoss EAP.
  • Ensure that you have your distributable Java application deployed to all the JBoss EAP servers in your cluster.

7.3.1. Cleaning Stale Discovery Files in Your Blob Container

If a JBoss EAP cluster that uses AZURE_PING is shut down abnormally, for example, using kill -9 to end the JBoss EAP process, some stale discovery files may be left in your blob container.

These files are usually cleaned up in a graceful cluster shutdown, but if left there from an abnormal shutdown, it may impact startup performance of cluster members attempting to contact nodes that are no longer online.

If this is a problem for you, you can set the following configuration to make the cluster coordinator remove and refresh all discovery files whenever the cluster view changes. 

/subsystem=jgroups/stack=udp/protocol=azure.AZURE_PING/property=remove_all_files_on_view_change:add(value=true)
Note

Alternatively, if cleaning your container on each view change is not ideal, you can reduce the number of join attempts for a node attempting to join a cluster. The default number of join attempts is 10. For example, to set the number of join attempts to 3: 

/subsystem=jgroups/stack=udp/protocol=pbcast.GMS/property=max_join_attempts:add(value=3)

The stale discovery files will still be present, but a node attempting to join a cluster will not spend as much time attempting to contact nodes that are no longer online.





Revised on 2018-10-11 12:30:55 UTC