8.4. Implement an HA Singleton

Summary

In JBoss EAP 5, HA singleton archives were deployed in the deploy-hasingleton/ directory separate from other deployments. This was done to prevent automatic deployment and to ensure the HASingletonDeployer service controlled the deployment and deployed the archive only on the master node in the cluster. There was no hot deployment feature, so redeployment required a server restart. Also, if the master node failed requiring another node to take over as master, the singleton service had to go through the entire deployment process in order to provide the service.

In JBoss EAP 6 this has changed. Using a SingletonService, the target service is installed on every node in the cluster but is only started on one node at any given time. This approach simplifies the deployment requirements and minimizes the time required to relocate the singleton master service between nodes.

Procedure 8.3. Implement an HA Singleton Service

  1. Write the HA singleton service application.

    The following is a simple example of a Service that is wrapped with the SingletonService decorater to be deployed as a singleton service.
    1. Create a singleton service.

      The following listing is an example of a singleton service:
      package com.mycompany.hasingleton.service.ejb;
      
      import java.util.concurrent.atomic.AtomicBoolean;
      import java.util.logging.Logger;
      
      import org.jboss.as.server.ServerEnvironment;
      import org.jboss.msc.inject.Injector;
      import org.jboss.msc.service.Service;
      import org.jboss.msc.service.ServiceName;
      import org.jboss.msc.service.StartContext;
      import org.jboss.msc.service.StartException;
      import org.jboss.msc.service.StopContext;
      import org.jboss.msc.value.InjectedValue;
      
      /**
       * @author <a href="mailto:wfink@redhat.com">Wolf-Dieter Fink</a>
       */
      public class EnvironmentService implements Service<String> {
          private static final Logger LOGGER = Logger.getLogger(EnvironmentService.class.getCanonicalName());
          public static final ServiceName SINGLETON_SERVICE_NAME = ServiceName.JBOSS.append("quickstart", "ha", "singleton");
          /**
           * A flag whether the service is started.
           */
          private final AtomicBoolean started = new AtomicBoolean(false);
      
          private String nodeName;
      
          private final InjectedValue<ServerEnvironment> env = new InjectedValue<ServerEnvironment>();
      
          public Injector<ServerEnvironment> getEnvInjector() {
              return this.env;
          }
      
          /**
           * @return the name of the server node
           */
          public String getValue() throws IllegalStateException, IllegalArgumentException {
              if (!started.get()) {
                  throw new IllegalStateException("The service '" + this.getClass().getName() + "' is not ready!");
              }
              return this.nodeName;
          }
      
          public void start(StartContext arg0) throws StartException {
              if (!started.compareAndSet(false, true)) {
                  throw new StartException("The service is still started!");
              }
              LOGGER.info("Start service '" + this.getClass().getName() + "'");
              this.nodeName = this.env.getValue().getNodeName();
          }
      
          public void stop(StopContext arg0) {
              if (!started.compareAndSet(true, false)) {
                  LOGGER.warning("The service '" + this.getClass().getName() + "' is not active!");
              } else {
                  LOGGER.info("Stop service '" + this.getClass().getName() + "'");
              }
          }
      }
      
      
    2. Create a singleton EJB to start the service as a SingletonService at server start.

      The following listing is an example of a singleton EJB that startes a SingletonService on server start:
      package com.mycompany.hasingleton.service.ejb;
      
      import java.util.Collection;
      import java.util.EnumSet;
      
      import javax.annotation.PostConstruct;
      import javax.annotation.PreDestroy;
      import javax.ejb.Singleton;
      import javax.ejb.Startup;
      
      import org.jboss.as.clustering.singleton.SingletonService;
      import org.jboss.as.server.CurrentServiceContainer;
      import org.jboss.as.server.ServerEnvironment;
      import org.jboss.as.server.ServerEnvironmentService;
      import org.jboss.msc.service.AbstractServiceListener;
      import org.jboss.msc.service.ServiceController;
      import org.jboss.msc.service.ServiceController.Transition;
      import org.jboss.msc.service.ServiceListener;
      import org.slf4j.Logger;
      import org.slf4j.LoggerFactory;
      
      
      /**
       * A Singleton EJB to create the SingletonService during startup.
       * 
       * @author <a href="mailto:wfink@redhat.com">Wolf-Dieter Fink</a>
       */
      @Singleton
      @Startup
      public class StartupSingleton {
        private static final Logger LOGGER = LoggerFactory.getLogger(StartupSingleton.class);
      
        /**
         * Create the Service and wait until it is started.<br/>
         * Will log a message if the service will not start in 10sec. 
         */
        @PostConstruct
        protected void startup() {
          LOGGER.info("StartupSingleton will be initialized!");
      
          EnvironmentService service = new EnvironmentService();
          SingletonService<String> singleton = new SingletonService<String>(service, EnvironmentService.SINGLETON_SERVICE_NAME);
          // if there is a node where the Singleton should deployed the election policy might set,
          // otherwise the JGroups coordinator will start it
          //singleton.setElectionPolicy(new PreferredSingletonElectionPolicy(new NamePreference("node2/cluster"), new SimpleSingletonElectionPolicy()));
          ServiceController<String> controller = singleton.build(CurrentServiceContainer.getServiceContainer())
              .addDependency(ServerEnvironmentService.SERVICE_NAME, ServerEnvironment.class, service.getEnvInjector())
              .install();
      
          controller.setMode(ServiceController.Mode.ACTIVE);
          try {
            wait(controller, EnumSet.of(ServiceController.State.DOWN, ServiceController.State.STARTING), ServiceController.State.UP);
            LOGGER.info("StartupSingleton has started the Service");
          } catch (IllegalStateException e) {
            LOGGER.warn("Singleton Service {} not started, are you sure to start in a cluster (HA) environment?",EnvironmentService.SINGLETON_SERVICE_NAME);
          }
        }
      
        /**
         * Remove the service during undeploy or shutdown
         */
        @PreDestroy
        protected void destroy() {
          LOGGER.info("StartupSingleton will be removed!");
          ServiceController<?> controller = CurrentServiceContainer.getServiceContainer().getRequiredService(EnvironmentService.SINGLETON_SERVICE_NAME);
          controller.setMode(ServiceController.Mode.REMOVE);
          try {
            wait(controller, EnumSet.of(ServiceController.State.UP, ServiceController.State.STOPPING, ServiceController.State.DOWN), ServiceController.State.REMOVED);
          } catch (IllegalStateException e) {
            LOGGER.warn("Singleton Service {} has not be stopped correctly!",EnvironmentService.SINGLETON_SERVICE_NAME);
          }
        }
      
        private static <T> void wait(ServiceController<T> controller, Collection<ServiceController.State> expectedStates, ServiceController.State targetState) {
          if (controller.getState() != targetState) {
            ServiceListener<T> listener = new NotifyingServiceListener<T>();
            controller.addListener(listener);
            try {
              synchronized (controller) {
                int maxRetry = 2;
                while (expectedStates.contains(controller.getState()) && maxRetry > 0) {
                  LOGGER.info("Service controller state is {}, waiting for transition to {}", new Object[] {controller.getState(), targetState});
                  controller.wait(5000);
                  maxRetry--;
                }
              }
            } catch (InterruptedException e) {
              LOGGER.warn("Wait on startup is interrupted!");
              Thread.currentThread().interrupt();
            }
            controller.removeListener(listener);
            ServiceController.State state = controller.getState();
            LOGGER.info("Service controller state is now {}",state);
            if (state != targetState) {
              throw new IllegalStateException(String.format("Failed to wait for state to transition to %s.  Current state is %s", targetState, state), controller.getStartException());
            }
          }
        }
      
        private static class NotifyingServiceListener<T> extends AbstractServiceListener<T> {
          @Override
          public void transition(ServiceController<? extends T> controller, Transition transition) {
            synchronized (controller) {
              controller.notify();
            }
          }
        }
      }
      
      
    3. Create a Stateless Session Bean to access the service from a client.

      The following is an example of a stateless session bean that accesses the service from a client:
      package com.mycompany.hasingleton.service.ejb;
      
      import javax.ejb.Stateless;
      
      import org.jboss.as.server.CurrentServiceContainer;
      import org.jboss.msc.service.ServiceController;
      import org.slf4j.Logger;
      import org.slf4j.LoggerFactory;
      
      /**
       * A simple SLSB to access the internal SingletonService.
       * 
       * @author <a href="mailto:wfink@redhat.com">Wolf-Dieter Fink</a>
       */
      @Stateless
      public class ServiceAccessBean implements ServiceAccess {
          private static final Logger LOGGER = LoggerFactory.getLogger(ServiceAccessBean.class);
      
          public String getNodeNameOfService() {
              LOGGER.info("getNodeNameOfService() is called()");
              ServiceController<?> service = CurrentServiceContainer.getServiceContainer().getService(
                      EnvironmentService.SINGLETON_SERVICE_NAME);
              LOGGER.debug("SERVICE {}", service);
              if (service != null) {
                  return (String) service.getValue();
              } else {
                  throw new IllegalStateException("Service '" + EnvironmentService.SINGLETON_SERVICE_NAME + "' not found!");
              }
          }
      }
      
      
    4. Create the business logic interface for the SingletonService.

      The following is an example of a business logic interface for the SingletonService:
      package com.mycompany.hasingleton.service.ejb;
      
      import javax.ejb.Remote;
      
      /**
       * Business interface to access the SingletonService via this EJB
       * 
       * @author <a href="mailto:wfink@redhat.com">Wolf-Dieter Fink</a>
       */
      @Remote
      public interface ServiceAccess {
          public abstract String getNodeNameOfService();
      } 
      
      
      
  2. Start each JBoss EAP 6 instance with clustering enabled.

    The method for enabling clustering depends on whether the servers are standalone or running in a managed domain.
    1. Enable clustering for servers running in a managed domain.

      You can enable clustering using the Management CLI or you can manually edit the configuration file.
      • Enable clustering using the Management CLI.

        1. Start your domain controller.

        2. Open a command prompt for your operating system.

        3. Connect to the Management CLI passing the domain controller IP address or DNS name.

          In this example, assume the IP address of the domain controller is 192.168.0.14.
          • For Linux, enter the following at the command line:
            $ EAP_HOME/bin/jboss-cli.sh --connect --controller=192.168.0.14
            
          • For Windows, enter the following at a command line:
            C:\>EAP_HOME\bin\jboss-cli.bat --connect --controller=192.168.0.14
            
          You should see the following response:
          Connected to domain controller at 192.168.0.14
        4. Add the main-server server group.

          [domain@192.168.0.14:9999 /] /server-group=main-server-group:add(profile="ha",socket-binding-group="ha-sockets") 
          {
              "outcome" => "success",
              "result" => undefined,
              "server-groups" => undefined
          }
          
        5. Create a server named server-one and add it to the main-server server group.

          [domain@192.168.0.14:9999 /]  /host=station14Host2/server-config=server-one:add(group=main-server-group,auto-start=false)
          {
              "outcome" => "success",
              "result" => undefined
          }
          
        6. Configure the JVM for the main-server server group.

          [domain@192.168.0.14:9999 /] /server-group=main-server-group/jvm=default:add(heap-size=64m,max-heap-size=512m)
          {
              "outcome" => "success",
              "result" => undefined,
              "server-groups" => undefined
          }
          
        7. Create a server named server-two, put it in a separate server group, and set its port offset to 100.

          [domain@192.168.0.14:9999 /]  /host=station14Host2/server-config=server-two:add(group=distinct2,socket-binding-port-offset=100)
          {
              "outcome" => "success",
              "result" => undefined
          }
          
      • Enable clustering by manually editing the server configuration files.

        1. Stop the JBoss EAP 6 server.

          Important

          You must stop the server before editing the server configuration file for your change to be persisted on server restart.
        2. Open the domain.xml configuration file for editing

          Designate a server group to use the ha profile and ha-sockets socket binding group as follows:
          <server-groups>
            <server-group name="main-server-group" profile="ha">
              <jvm name="default">
                <heap size="64m" max-size="512m"/>
              </jvm>
              <socket-binding-group ref="ha-sockets"/>
            </server-group>
          </server-groups>
          
          
        3. Open the host.xml configuration file for editing

          Modify the file as follows:
          <servers>
            <server name="server-one" group="main-server-group" auto-start="false"/>
            <server name="server-two" group="distinct2">
              <socket-bindings port-offset="100"/>
            </server>
          <servers>
          
          
        4. Start the server.

          • For Linux, type: EAP_HOME/bin/domain.sh
          • For Microsoft Windows, type: EAP_HOME\bin\domain.bat
    2. Enable clustering for standalone servers

      To enable clustering for standalone servers, start the server using the node name and the standalone-ha.xml configuration file as follows:
      • For Linux, type: EAP_HOME/bin/standalone.sh --server-config=standalone-ha.xml -Djboss.node.name=UNIQUE_NODE_NAME
      • For Microsoft Windows, type: EAP_HOME\bin\standalone.bat --server-config=standalone-ha.xml -Djboss.node.name=UNIQUE_NODE_NAME

    Note

    To avoid port conflicts when running multiple servers on one machine, configure the standalone-ha.xml file for each server instance to bind on a separate interface. Alternatively, you can start subsequent server instances with a port offset using an argument like the following on the command line: -Djboss.socket.binding.port-offset=100.
  3. Deploy the application to the servers

    If you use Maven to deploy your application, use the following Maven command to deploy to the server running on the default ports:
    mvn clean install jboss-as:deploy
    To deploy to additional servers, pass the server name and port number on the command line:
    mvn clean package jboss-as:deploy -Ddeploy.hostname=localhost -Ddeploy.port=10099