Chapter 4. Process Server clusters in a runtime environment

In a runtime environment, Process Server runs services that contain rules and processes that support business decisions. The primary benefit of clustering a Process Server runtime environment is load balancing. If activity on one node of the cluster increases, that activity can be shared among the other nodes of the cluster to improve performance.

To create a Process Server clustered runtime environment, you download and extract Red Hat JBoss EAP 7.1 and Process Server. Then, you configure Red Hat JBoss EAP 7.1 for a domain-mode cluster, start the cluster, and install Process Server on each cluster node.

If desired, you can then cluster the headless Process Automation Manager controller and Smart Router.

4.1. Downloading and extracting Red Hat JBoss EAP 7.1 and Process Server

Complete the steps in this section to download and install Red Hat JBoss EAP 7.1 and download and repackage Process Server for installation in a clustered environment.

Procedure

  1. Install Red Hat JBoss EAP 7.1 on each node of the cluster:

    1. Navigate to the Software Downloads page in the Red Hat Customer Portal (login required) and select the product and version from the drop-down options:

      • Product: Enterprise Application Platform
      • Version: 7.1
    2. Click Download next to Red Hat JBoss Enterprise Application Platform 7.1.0. (jboss-eap-7.1.0.zip).
    3. Extract the jboss-eap-7.1.0.zip file. The jboss-eap-7.1/jboss-eap-7.1 directory is referred to as EAP_HOME.
  2. Download and extract Process Server:

    1. Navigate to the Software Downloads page in the Red Hat Customer Portal and select the product and version from the drop-down options:

      • Product: Process Automation Manager
      • Version: 7.1
      • Download Red Hat Process Automation Manager 7.1.0 Process Server for All Supported EE7 Containers (rhpam-7.1.0-kie-server-ee7.zip).
    2. Extract the rhpam-7.1.0-kie-server-ee7.zip archive to a temporary directory. In the following examples this directory is called TEMP_DIR.
  3. Repackage the kie-server.war directory:

    1. Navigate to the TEMP_DIR/rhpam-7.1.0-kie-server-ee7/kie-server.war directory.
    2. Select the contents of the TEMP_DIR/rhpam-7.1.0-kie-server-ee7/kie-server.war directory and create the kie-server.zip file.
    3. Rename kie-server.zip to kie-server.war. This is the file that you will use to deploy Process Server on the cluster nodes.
    4. If desired, copy the new kie-server.war file to a location that is more convenient to deploy from.
  4. If you want to use a security manager with Process Server, copy the contents of the TEMP_DIR/rhpam-7.1.0-kie-server-ee7/SecurityPolicy directory to the EAP_HOME/bin directory on each node of the cluster.

4.2. Configuring a Red Hat JBoss EAP 7.1 domain-mode cluster for Process Server

To centrally manage and publish configurations for the servers across a cluster, configure the Red Hat JBoss EAP cluster in domain mode for Process Server. Domain mode provides a central location to store and publish configurations.

Procedure

  1. Install the JDBC driver on all Red Hat JBoss EAP instances that are part of this cluster. For more information, see the "JDBC Drivers" section of the Red Hat JBoss EAP 7.1 Configuration Guide.
  2. On the the master node of the cluster (where you will run the domain controller), open the EAP_HOME/domain/configuration/domain.xml file in a text editor.
  3. Make the following changes for each server group in the server-groups element:

    1. Specify the server-group name and jvm name.
    2. To enable high availability, change the profile to full-ha and the socket-binding-group to full-ha-sockets.

      <server-group name="some-server-group" profile="full-ha">
          <jvm name="some-jvm">
              <heap size="1000m" max-size="1000m"/>
          </jvm>
          <socket-binding-group ref="full-ha-sockets"/>
      </server-group>
  4. Edit the data-stores property and the timer-service thread-pool-name above it:

    • The datasource-jndi-name is the JNDI name of the database specified at the beginning of this procedure.
    • You can enter any name for the value of the partition property. However, a node will only see timers from other nodes that have the same partition name. Grouping nodes into partitions by assigning partition names enables you to break a large cluster up into several smaller clusters. Doing this should improve performance. For example, instead of having a cluster of 100 nodes, where all 100 nodes are trying to execute and refresh the same timers, you can create 20 clusters of 5 nodes by giving every group of 5 a different partition name.
    • Replace the default-data-store attribute value with ejb_timer_ds.
    • Set the value of refresh-interval in milliseconds to specify how often the EJB timer connects to the database to synchronize and load tasks to be processed.

      <timer-service thread-pool-name="default" default-data-store="ejb_timer_ds">
      <data-stores>
          <database-data-store name="ejb_timer_ds" datasource-jndi-name="java:jboss/datasources/ejb_timer" database="postgresql" partition="ejb_timer_part" refresh-interval="30000"/>
      </data-stores>
      </timer-service>

      The following table lists the supported databases and the corresponsing database attribute value:

      DatabaseAttribute value

      Hyper SQL (for demonstration purposes only, not supported)

      hsql

      PostgreSQL

      postgresql

      Oracle

      oracle

      IBM DB2

      db2

      Microsoft SQL Server

      mssql

      MySQL and MariaDB

      mysql

  5. Add the Process Server and EJB timer data sources to the full-ha profile. In these examples, <DATABASE> is the name of the database, <SERVER_NAME> is the host name of the JNDI database, and <USER_NAME> and <USER_PWD> are the credentials for that database.

    • Add the data source to allow Process Server to connect to the database, for example:

      <xa-datasource jndi-name="java:/jboss/datasources/rhpam" pool-name="rhpam-RHPAM" use-java-context="true" enabled="true">
        <xa-datasource-property name="DatabaseName"><DATABASE></xa-datasource-property>
        <xa-datasource-property name="PortNumber">5432</xa-datasource-property>
        <xa-datasource-property name="ServerName"><SERVER_NAME></xa-datasource-property>
        <driver>postgresql</driver>
        <security>
          <user-name><USER_NAME></user-name>
          <password><USER_PWD></password>
      </security>
      </xa-datasource>
    • Add the data source to enable the EJB timer, for example:

      <xa-datasource jndi-name="java:jboss/datasources/ejb_timer" pool-name="ejb_timer" use-java-context="true" enabled="true">
          <xa-datasource-property name="DatabaseName"><DATABASE></xa-datasource-property>
          <xa-datasource-property name="PortNumber">5432</xa-datasource-property>
          <xa-datasource-property name="ServerName"><SERVER_NAME></xa-datasource-property>
          <driver>postgresql</driver>
          <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
          <security>
              <user-name><USER_NAME></user-name>
              <password><USER_PWD></password>
          </security>
      </xa-datasource>
      Warning

      You must use two different databases for Process Server runtime data and EJB timer data.

  6. In the <subsystem xmlns="urn:jboss:domain:messaging-activemq:2.0"> element, make the following changes:

    1. Replace <cluster password="${jboss.messaging.cluster.password:CHANGE ME!!}"/> with <cluster user="<CLUSTER_USER>" password="<CLUSTER_PWD>"/> where <CLUSTER_USER> is the user used by Red Hat JBoss EAP to form a messaging cluster and <CLUSTER_PWD> is the password for that user.
    2. In the security-setting element add the kie-server role:

      <subsystem xmlns="urn:jboss:domain:messaging-activemq:2.0">
          <server name="default">
              <security-setting name="#">
                  <role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
                  <role name="kie-server" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
              </security-setting>
  7. Add the following properties to the <system-properties> element and replace the following placeholders:

    • <JNDI_NAME> is the JNDI name of your data source. For Red Hat Process Automation Manager, this is java:/jboss/datasources/rhpam.
    • <DIALECT> is the hibernate dialect for your database.

      The following dialects are supported:

      • DB2: org.hibernate.dialect.DB2Dialect
      • MSSQL: org.hibernate.dialect.SQLServer2012Dialect
      • MySQL: org.hibernate.dialect.MySQL5InnoDBDialect
      • MariaDB: org.hibernate.dialect.MySQL5InnoDBDialect
      • Oracle: org.hibernate.dialect.Oracle10gDialect
      • PostgreSQL: org.hibernate.dialect.PostgreSQL82Dialect
      • PostgreSQL plus: org.hibernate.dialect.PostgresPlusDialect

        <system-properties>
          <property name="org.kie.server.persistence.ds" value="<JNDI_NAME>"/>
          <property name="org.kie.server.persistence.dialect" value="<DIALECT>"/>
          <property name="org.jbpm.ejb.timer.tx" value="true"/>
        </system-properties>
  8. Save the domain.xml file.

4.3. Configuring Red Hat JBoss EAP 7.1 host files for a Process Server clustered environment

After you have installed Red Hat JBoss EAP and configured the domain.xml file for a Process Server clustered environment, you must configure the host.xml and host-slave.xml files.

Prerequisite

Red Hat JBoss EAP 7.1 is installed on each node of the cluster.

Procedure

  1. Complete the following steps on the master node of the cluster:

    1. Open the EAP_HOME/domain/configuration/host.xml file in a text editor.
    2. In the main-server-group <servers> element, add the servers that will be part of the cluster.
  2. Complete the following steps in the host.xml file on the master node and in the host-slave.xml file on each slave node:

    1. Open the EAP_HOME/domain/configuration/host-slave.xml file in a text editor.
    2. In the main-server-group <servers> element, add the servers that will be part of the cluster.

4.4. Starting the cluster and distributing Process Server

After you have configured a Red Hat JBoss EAP cluster and repackaged Process Server, you must start the cluster and distribute the repackaged kie-server.war file.

Prerequisites

  • A Red Hat JBoss EAP 7.1 cluster is available and the domain.xml, host.xml, and host-slave.xml files have been configured.
  • The repackaged kie-server.war file is available.

Procedure

  1. Add Red Hat JBoss EAP management users to the master node (where you configured the domain.xml file) as described in the Red Hat JBoss EAP 7.1 Configuration Guide.
  2. On each node of the cluster, add Process Server users as described in Installing and configuring Red Hat Process Automation Manager on Red Hat JBoss EAP 7.1.

    1. To start the master node, enter one of the following commands:

      • On Linux or UNIX-based systems, enter:

        <EAP_HOME>/bin/domain.sh

        On Windows, enter:

        <EAP_HOME>/bin/domain.bat
    2. To start the slave nodes, on each of the slave nodes enter one of the following commands, where <MASTER_NODE_IP> is the IP address of the master node (where the domain controller is configured):

      • On Linux or UNIX-based systems, enter:

        <EAP_HOME>/bin/domain.sh --host-config=host-slave.xml -Djboss.domain.master.address=<MASTER_NODE_IP>
      • On Windows, enter:

        <EAP_HOME>/bin/domain.bat --host-config=host-slave.xml -Djboss.domain.master.address=<MASTER_NODE_IP>

        For more information, see the following sections of the Red Hat JBoss EAP 7.1 Configuration Guide:

      • "Starting JBoss EAP"
      • "Managed Domain Configuration Files".
  3. On each node of the cluster, review the EAP_HOME/domain/servers/SERVER_NAME/log/server.log file to verify that the nodes are available.
  4. To deploy the kie-server.war file that you created previously into the server group, complete the following steps on the master node:

    1. Log in to the Red Hat JBoss EAP Administration console of your domain as a management user.
    2. Click Deployments → Server Groups→ main-server-group and click Add.
    3. In the dialog box, click Upload a new deployment and click Next.
    4. In the Upload Deployments dialog box, click Browse, select the kie-server.war file, and click Next.
    5. Click Enable and click Next.

      Note

      Make sure to check deployment unit readiness with every cluster member.

      When a deployment unit is created on a cluster node, it takes some time before it is distributed among all cluster members. You can check the deployment status using the server Administration console or REST API. However, if the query is sent to the node where the deployment was originally issued, the query will return a value of deployed. If the query is sent to a node where the deployment has not yet been distributed, the query returns DeploymentNotFoundException.

  5. To verify that the Process Server main endpoint is available and business process management (BPM) capabilities are enabled, enter the following command and replace <node_IP> and <node_port> with the IP address and port number of the node you are verifying:

    curl -X GET "http://<node_IP>:<node_port>/kie-server/services/rest/server"

    If the following line is present in the output, BPM capabilities are enabled:

    <capabilities>BPM</capabilities>

For more information about installing Business Central, see Installing and configuring Red Hat Process Automation Manager on Red Hat JBoss EAP 7.1.

4.5. Clustering with the headless Process Automation Manager controller

The Process Automation Manager controller is integrated with Business Central. However, if you do not install Business Central, you can install the headless Process Automation Manager controller and use the REST API or the Process Server Java Client API to interact with it.

Prerequisites

  • A backed-up Red Hat JBoss EAP installation version 7.1 or later is available. The base directory of the Red Hat JBoss EAP installation is referred to as EAP_HOME.
  • Sufficient user permissions to complete the installation are granted.
  • An NFS server with a mounted partition is available.

Procedure

  1. Navigate to the Software Downloads page in the Red Hat Customer Portal (login required), and select the product and version from the drop-down options:

    • Product: Process Automation Manager
    • Version: 7.1
  2. Download Red Hat Process Automation Manager 7.1.0 Add Ons (the rhpam-7.1.0-add-ons.zip file).
  3. Unzip the rhpam-7.1.0-add-ons.zip file. The rhpam-7.1-controller-ee7.zip file is in the unzipped directory.
  4. Extract the rhpam-7.1-controller-ee7.zip file to a temporary directory. In the following examples this directory is called TEMP_DIR.
  5. Repackage the controller.war directory:

    1. Navigate to the TEMP_DIR/rhpam-7.1.0-add-ons/rhpam-7.1-controller-ee7/controller.war directory.
    2. Select the contents of the TEMP_DIR/rhpam-7.1.0-add-ons/rhpam-7.1-controller-ee7/controller.war directory and create the controller.zip file.
    3. Rename controller.zip to controller.war. This is the file that you will use to deploy the headless Process Automation Manager controller on the cluster nodes.
    4. If desired, copy the new controller.war file to a location that is more convenient to deploy from.
  6. If you want to use a security manager with the headless Process Automation Manager controller, copy the contents of the TEMP_DIR/rhpam-7.1.0-add-ons/rhpam-7.1-controller-ee7/SecurityPolicy directory to the EAP_HOME/bin directory on each node of the cluster.
  7. Add Red Hat JBoss EAP management users to the master node (where you configured the domain.xml file) as described in the Red Hat JBoss EAP 7.1 Configuration Guide.
  8. On each node of the cluster, add users for the headless Process Automation Manager controller as described in the "Installing the headless Process Automation Manager controller" section of Installing and configuring Red Hat Process Automation Manager on Red Hat JBoss EAP 7.1.
  9. Complete the following steps in the host.xml file on the master node and in the host-slave.xml file on each slave node:

    1. Open the EAP_HOME/domain/configuration/host.xml or EAP_HOME/domain/configuration/host-slave.xml file in a text editor.
    2. In the main-server-group <servers> element, add the servers that will be part of the cluster.
    3. Add the following properties to the <system-properties> element and replace <NFS_STORAGE> with the absolute path to the NFS storage where the template configuration is stored.

      <system-properties>
        <property name="org.kie.server.controller.templatefile.watcher.enabled" value="true"/>
        <property name="org.kie.server.controller.templatefile" value="<NFS_STORAGE>"/>
      </system-properties>

      If the value of the org.kie.server.controller.templatefile.watcher.enabled property is set to true, a separate thread is started to watch for modifications of the template file. The default interval for these checks is 30000 milliseconds and can be further controlled by the org.kie.server.controller.templatefile.watcher.interval system property. If the value of this property is set to false, changes to the template file are detected only when the server restarts.

  10. To deploy the controller.war file that you created previously into the server group, complete the following steps on the master node:

    1. Log in to the Red Hat JBoss EAP Administration console of your domain as a management user.
    2. Click Deployments → Server Groups→ main-server-group and click Add.
    3. In the dialog box, click Upload a new deployment and click Next.
    4. In the Upload Deployments dialog box, click Browse, select the controller.war file, and click Next.
    5. Click Enable and click Next.

      Note

      Make sure to check deployment unit readiness with every cluster member.

      When a deployment unit is created on a cluster node, it takes some time before it is distributed among all cluster members. You can check the deployment status using the server Administration console or REST API. However, if the query is sent to the node where the deployment was originally issued, the query will return a value of deployed. If the query is sent to a node where the deployment has not yet been distributed, the query returns DeploymentNotFoundException.

For more information about installing Business Central, see Installing and configuring Red Hat Process Automation Manager on Red Hat JBoss EAP 7.1.

4.6. Clustering Smart Router for a runtime environment

Smart Router enables you to aggregate multiple independent Process Server instances as though they are a single server. It performs the role of an intelligent load balancer because it can both route requests to individual Process Servers and aggregate data from different Process Servers. Through aliases, Smart Router is a proxy to Process Servers. Smart Router performs the following tasks:

  • Collects information from various server instances in a single client request
  • Finds the right server for a specific request
  • Aggregates responses from different servers
  • Provides efficient load-balancing
  • Manages changing environments, for example adding and removing server instances
  • Manages registration with the Process Automation Manager controller

This section describes how to install Smart Router and configure it for a Red Hat Process Automation Manager runtime environment.

Note

Load balancing requests for Smart router cluster must be managed externally, using standard load balancing tools.

Prerequisite

Process Server is installed on each node of a Red Hat JBoss EAP 7.1 cluster.

Procedure

  1. Navigate to the Software Downloads page in the Red Hat Customer Portal (login required), and select the product and version from the drop-down options:

    • Product: Process Automation Manager
    • Version: 7.1
  2. Download Red Hat Process Automation Manager 7.1.0 Add-Ons.
  3. Extract the downloaded rhpam-7.1.0-add-ons.zip file to a temporary directory. The rhpam-7.1-smart-router.jar file is in the extracted rhpam-7.1.0-add-ons directory.
  4. Copy the rhpam-7.1-smart-router.jar file to location where you will run the file.
  5. From the temporary directory, enter the following command to start SmartRouter:

    java
    -Dorg.kie.server.router.host=<ROUTER_HOST>
    -Dorg.kie.server.router.port=<ROUTER_PORT>
    -Dorg.kie.server.controller=<CONTROLLER_URL>
    -Dorg.kie.server.controller.user=<CONTROLLER_USER>
    -Dorg.kie.server.controller.pwd=<CONTROLLER_PWD>
    -Dorg.kie.server.router.config.watcher.enabled=true
    -Dorg.kie.server.router.repo=<NFS_STORAGE>
    -jar rhpam-7.1-smart-router.jar

    The properties in the preceding command have the following default values:

    org.kie.server.router.host=localhost
    org.kie.server.router.port=9000
    org.kie.server.controller= N/A
    org.kie.server.controller.user=kieserver
    org.kie.server.controller.pwd=kieserver1!
    org.kie.server.router.repo= <CURRENT_WORKING_DIR>
    org.kie.server.router.config.watcher.enabled=false
    Note

    The router is capable of providing an aggregate sort, however the data returned when used through the management console is in raw format. Therefore, sorting is in whatever way individual servers return.

    Paging is supported in its standard format.

  6. To use Smart Router from the client side, use the Smart Router URL instead of the Process Server URL, for example:

    KieServicesConfiguration config = KieServicesFactory.newRestConfiguration("http://smartrouter.example.com:9000", "USERNAME", "PASSWORD");

    In this example, smartrouter.example.com is the Smart Router URL, and USERNAME and PASSWORD are the log in credentials for the Smart Router configuration.