Fuse ESB - Clustering Guide - Pure Master/Slave

Overview

The pure master/slave failover pattern replicates data between a pair of brokers using a dedicated connection. The advantage of this approach is that it works independently of the persistence layer: it does not require a shared database or a shared file system.

Initial state of the master/slave topology

Figure 23 shows the initial state of the pure master/slave topology.

Figure 23. Pure Master/Slave Initial State

In this topology, the master broker is an ordinary broker with no special configuration. Typicaly, the master is configured with some form of persistent storage, but this is not a requirement.

The slave broker is configured specially with a master connector, which connects to the master broker in order to duplicate the data stored in the master. While the master/slave connection is active, the slave consumes all events from the master: including messages, acknowledgments, and transactional states. At this stage, the slave does not start any transport connectors or network connectors (even if these are configured). Its sole purpose is to duplicate the state of the master.

Producer and consumer clients are configured with a failover URL that tries to connect first of all to the master broker (see Configuring clients of the master/slave pair).

The master broker will respond to a client only after a message exchange has been successfully passed to the slave. For example, a commit in a client transaction will not complete until both the master and the slave have processed the commit.

After failure of the master

In the event of the master failing, the slave supports two alternative modes of operation, as follows:

Slave broker takes over—in this case, the slave starts up all of its transport connectors and network connectors and takes the place of the master broker. Clients must be configured to fail over to the slave.
Slave broker closes down—in this case, there is no longer any broker available. The sole purpose of the slave is to preserve a copy of the messaging data, in case there is a catastrophic hardware failure on the master host.

Figure 24 shows the state of the system after the master broker has failed, assuming that the slave broker takes over from the master.

Figure 24. Pure Master/Slave after Master Failure

Limitations of the pure master/slave topology

The pure master/slave topology is subject to the following limitations:

Only one slave can be connected to the master.
A failed master cannot be re-introduced without shutting down the the slave broker (no automatic failback).
There is no automatic store synchronization between the brokers after a failure. Replicating the data from the slave back to the master is a manual process.

Configuring the master

No special configuration is required for the master, although there are a couple of optional attributes you can set on the master's broker element. Example 11 shows a sample configuration for a master broker in the pure master/slave topology.

Example 11. Master Configuration for Pure Master/Slave

<broker brokerName="master"
        waitForSlave="true"
        shutdownOnSlaveFailure="false"
        xmlns="http://activemq.apache.org/schema/core">
  ...
  <transportConnectors>
  	  <transportConnector uri="tcp://masterhost:61616"/>
  </transportConnectors>
  ...
</broker>

You can optionally set any of the following attributes on the broker element of a master broker:

waitForSlave: If true, the master will wait untill a slave has attached before completing its start-up sequence. Default is false.
shutdownOnSlaveFailure: If true, the master shuts down, if the slave connection is lost, thereby ensuring that the master does not get out of sync with the slave. Default is false.

Note

You should not configure a network connector between the master and its slave. Only the master connector should be created (in the slave's configuration). If you explicitly configure a network connector, you may encounter race conditions when the master broker is under heavy load.

Configuring the slave

There are two alternative approaches to configuring the slave. Example 12 shows how to configure the slave using the masterConnector element, which enables you to specify username and password credentials for the connection.

Example 12. Slave Configuration for Pure Master/Slave

<broker brokerName="slave"
        xmlns="http://activemq.apache.org/schema/core">
  ...
  <services>
    <masterConnector
      remoteURI="tcp://localhost:62001"
      userName="James"
      password="Cheese"/>
  </services>

  <transportConnectors>
    <transportConnector uri="tcp://slavehost:61616"/>
  </transportConnectors>
  ...
</broker>

The masterConnector element supports the following attributes:

remoteURI: Specifies the address of the master's transport connector port.
userName: Username for connecting to the master (if authentication is enabled).
password: Password for connecting to the master (if authentication is enabled).

Example 13 shows the alternative approach to configuration, by setting attributes on the broker element. This approach does not support setting credentials, however.

Example 13. Alternative Slave Configuration

<broker brokerName="slave"
        masterConnectorURI="tcp://masterhost:62001"
        shutdownOnMasterFailure="false"
        xmlns="http://activemq.apache.org/schema/core">
  ...
  <transportConnectors>
  	  <transportConnector uri="tcp://slavehost:61616"/>
  </transportConnectors>
  ...
</broker>

The masterConnector element supports the following attributes:

masterConnectorURI

Specifies the address of the master's transport connector port.

shutdownOnMasterFailure

If true, the slave shuts down when the master fails; otherwise, the slave takes over as the new master. Default is false.

The slave ensures that there is a separate copy of each message and acknowledgment on another machine, which can protect against catastrophic hardware failure. If the master fails, you might want the slave to shut down as well, if protection against data loss is your main priority. You can then manually duplicate the slave's copy of the data before restoring the system.

Recovery procedure

Recovery after master failure is a manual process. Perform the following steps:

Shut down the slave broker.
Note
Clients do not need to be restarted. If they are failover clients, they will automatically reconnect when the master/slave topology is restored.
Copy the slave's data directory over to the master's data directory.
Start the master and the slave.