Chapter 11. JGroups Subsystem Tuning

For optimal network performance it is recommended that you use UDP multicast for JGroups in environments that support it.

Note

TCP has more overhead and is often considered slower than UDP since it handles error checking, packet ordering, and congestion control itself. JGroups handles these features for UDP, whereas TCP guarantees them itself. TCP is a good choice when using JGroups on unreliable or high congestion networks, or when multicast is not available.

This chapter assumes that you have chosen your JGroups stack transport protocol (UDP or TCP) and communications protocols that JGroups cluster communications will use. See the JBoss EAP Configuration Guide for more information about cluster communication with JGroups.

11.1. Monitoring JGroups Statistics

You can enable statistics for the jgroups subsystem to monitor JBoss EAP clustering using the management CLI or through JMX.

Note

Enabling statistics adversely affects performance. Only enable statistics when necessary.

  1. Use the following command to enable statistics for a JGroups channel.

    Note

    In a managed domain, precede these commands with /profile=PROFILE_NAME.

    /subsystem=jgroups/channel=CHANNEL_NAME:write-attribute(name=statistics-enabled,value=true)

    For example, use the following command to enable statistics for the default ee channel.

    /subsystem=jgroups/channel=ee:write-attribute(name=statistics-enabled,value=true)
  2. Reload the JBoss EAP server.

    :reload
  3. You can now see JGroups statistics using either the management CLI, or through JMX with a JVM monitoring tool:

    • To use the management CLI, use the :read-resource(include-runtime=true) command on the JGroups channel or protocol that you want to see the statistics for.

      Note

      In a managed domain, precede these commands with /host=HOST_NAME/server=SERVER_NAME.

      For example:

      • To see the statistics for the ee channel, use the following command:

        /subsystem=jgroups/channel=ee:read-resource(include-runtime=true)
      • To see the statistics for the FD_ALL protocol in the ee channel, use the following command:

        /subsystem=jgroups/channel=ee/protocol=FD_ALL:read-resource(include-runtime=true)
    • To connect to JBoss EAP using a JVM monitoring tool, see the Monitoring Performance chapter. You can see the statistics on JGroups MBeans through the JMX connection.

11.2. Networking and Jumbo Frames

Where possible, it is recommended that the network interface for JGroups traffic should be part of a dedicated Virtual Local Area Network (VLAN). This allows you to separate cluster communications from other JBoss EAP network traffic to more easily control cluster network performance, throughput, and security.

Another network configuration to consider to improve cluster performance is to enable jumbo frames. If your network environment supports it, enabling jumbo frames by increasing the Maximum Transmission Unit (MTU) can help boost network performance, especially in high throughput environments.

To use jumbo frames, all NICs and switches in your network must support it. See the Red Hat Customer Portal for instructions on enabling jumbo frames for Red Hat Enterprise Linux.

11.3. Message Bundling

Message bundling in JGroups improves network performance by assembling multiple small messages into larger bundles. Rather than sending out many small messages over the network to cluster nodes, instead messages are queued until the maximum bundle size is reached or there are no more messages to send. The queued messages are assembled into a larger message bundle and then sent.

This bundling reduces communications overhead, especially in TCP environments where there is a higher overhead for network communications.

Configuring Message Bundling

JGroups message bundling is configured using the max_bundle_size property. The default max_bundle_size is 64KB.

The performance improvements of tuning the bundle size depend on your environment, and whether more efficient network traffic is balanced against a possible delay of communications while the bundle is assembled.

Use the following management CLI command to configure max_bundle_size.

/subsystem=jgroups/stack=STACK_NAME/transport=TRANSPORT_TYPE/property=max_bundle_size:add(value=BUNDLE_SIZE)

For example, to set max_bundle_size to 60K for the default udp stack:

/subsystem=jgroups/stack=udp/transport=UDP/property=max_bundle_size:add(value=60K)

11.4. JGroups Thread Pools

The jgroups subsystem uses its own thread pools for processing cluster communication. JGroups contains thread pools for default, internal, oob, and timer functions which you can configure individually. Each JGroups thread pool includes configurable attributes for keepalive-time, max-threads, min-threads, and queue-length.

Appropriate values for each thread pool attribute depend on your environment, but for most situations the default values should suffice.

See the JBoss EAP Configuration Guide for instructions on how to configure JGroups thread pools.

11.5. JGroups Send and Receive Buffers

The jgroups subsystem has configurable send and receive buffers for both UDP and TCP stacks.

Appropriate values for JGroups buffers depend on your environment, but for most situations the default values should suffice. It is recommended that you test your cluster under load in a development environment to tune appropriate values for the buffer sizes.

Note

Your operating system may limit the available buffer sizes and JBoss EAP may not be able to use its configured buffer values. See Resolving Buffer Size Warnings in the JBoss EAP Configuration Guide.

See the JBoss EAP Configuration Guide for instructions on how to configure JGroups send and receive buffers.