Show Table of Contents
30.2. Configure JGroups (Library Mode)
Red Hat JBoss Data Grid must have an appropriate JGroups configuration in order to operate in clustered mode.
Example 30.5. JGroups XML Configuration
<infinispan xmlns="urn:infinispan:config:8.3">
<jgroups>
<stack-file name="jgroupsStack" path="/path/to/jgroups/xml/jgroups.xml}"/>
</jgroups>
<cache-container name="default" default-cache="default">
<transport stack="jgroupsStack" lock-timeout="600000" cluster="default" />
</cache-container>
</infinispan>
JBoss Data Grid will first search for
jgroups.xml in the classpath; if no instances are found in the classpath it will then search for an absolute path name.
30.2.1. JGroups Transport Protocols
A transport protocol is the protocol at the bottom of a protocol stack. Transport Protocols are responsible for sending and receiving messages from the network.
Red Hat JBoss Data Grid ships with both UDP and TCP transport protocols.
30.2.1.1. The UDP Transport Protocol
UDP is a transport protocol that uses:
- IP multicasting to send messages to all members of a cluster.
- UDP datagrams for unicast messages, which are sent to a single member.
When the UDP transport is started, it opens a unicast socket and a multicast socket. The unicast socket is used to send and receive unicast messages, the multicast socket sends and receives multicast sockets. The physical address of the channel will be the same as the address and port number of the unicast socket.
30.2.1.2. The TCP Transport Protocol
TCP/IP is a replacement transport for UDP in situations where IP multicast cannot be used, such as operations over a WAN where routers may discard IP multicast packets.
TCP is a transport protocol used to send unicast and multicast messages.
- When sending multicast messages, TCP sends multiple unicast messages.
As IP multicasting cannot be used to discover initial members, another mechanism must be used to find initial membership.
30.2.1.3. Using the TCPPing Protocol
Some networks only allow TCP to be used. The pre-configured
default-configs/default-jgroups-tcp.xml includes the MPING protocol, which uses UDP multicast for discovery. When UDP multicast is not available, the MPING protocol, has to be replaced by a different mechanism. The recommended alternative is the TCPPING protocol. The TCPPING configuration contains a static list of IP addresses which are contacted for node discovery.
Example 30.6. Configure the JGroups Subsystem to Use TCPPING
<TCP bind_port="7800" />
<TCPPING initial_hosts="${jgroups.tcpping.initial_hosts:HostA[7800],HostB[7801]}"
port_range="1" />30.2.2. Pre-Configured JGroups Files
Red Hat JBoss Data Grid ships with a number of pre-configured JGroups files packaged in
infinispan-embedded.jar, and are available on the classpath by default. In order to use one of these files, specify one of these file names instead of using jgroups.xml.
The JGroups configuration files shipped with JBoss Data Grid are intended to be used as a starting point for a working project. JGroups will usually require fine-tuning for optimal network performance.
The available configurations are:
default-configs/default-jgroups-udp.xmldefault-configs/default-jgroups-tcp.xmldefault-configs/default-jgroups-ec2.xmldefault-configs/default-jgroups-google.xml
30.2.2.1. default-jgroups-udp.xml
The
default-configs/default-jgroups-udp.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-udp.xml configuration
- uses UDP as a transport and UDP multicast for discovery.
- is suitable for large clusters (over 8 nodes).
- is suitable if using Invalidation or Replication modes.
The behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
Table 30.1. default-jgroups-udp.xml System Properties
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.udp.mcast_addr | IP address to use for multicast (both for communications and discovery). Must be a valid Class D IP address, suitable for IP multicast. | 228.6.7.8 | No |
| jgroups.udp.mcast_port | Port to use for multicast socket | 46655 | No |
| jgroups.udp.ip_ttl | Specifies the time-to-live (TTL) for IP multicast packets. The value here refers to the number of network hops a packet is allowed to make before it is dropped | 2 | No |
30.2.2.2. default-jgroups-tcp.xml
The
default-configs/default-jgroups-tcp.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-tcp.xml configuration
- uses TCP as a transport and UDP multicast for discovery.
- is generally only used where multicast UDP is not an option.
- TCP does not perform as well as UDP for clusters of eight or more nodes. Clusters of four nodes or fewer result in roughly the same level of performance for both UDP and TCP.
As with other pre-configured JGroups files, the behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
Table 30.2. default-jgroups-tcp.xml System Properties
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.tcp.address | IP address to use for the TCP transport. | 127.0.0.1 | No |
| jgroups.tcp.port | Port to use for TCP socket | 7800 | No |
| jgroups.mping.mcast_addr | IP address to use for multicast (for discovery). Must be a valid Class D IP address, suitable for IP multicast. | 228.6.7.8 | No |
| jgroups.mping.mcast_port | Port to use for multicast socket | 46655 | No |
| jgroups.udp.ip_ttl | Specifies the time-to-live (TTL) for IP multicast packets. The value here refers to the number of network hops a packet is allowed to make before it is dropped | 2 | No |
30.2.2.3. default-jgroups-ec2.xml
The
default-configs/default-jgroups-ec2.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-ec2.xml configuration
- uses TCP as a transport and S3_PING for discovery.
- is suitable on Amazon EC2 nodes where UDP multicast isn't available.
As with other pre-configured JGroups files, the behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
Table 30.3. default-jgroups-ec2.xml System Properties
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.tcp.address | IP address to use for the TCP transport. | 127.0.0.1 | No |
| jgroups.tcp.port | Port to use for TCP socket | 7800 | No |
| jgroups.s3.access_key | The Amazon S3 access key used to access an S3 bucket | Yes | |
| jgroups.s3.secret_access_key | The Amazon S3 secret key used to access an S3 bucket | Yes | |
| jgroups.s3.bucket | Name of the Amazon S3 bucket to use. Must be unique and must already exist | Yes | |
| jgroups.s3.pre_signed_delete_url | The pre-signed URL to be used for the DELETE operation. | Yes | |
| jgroups.s3.pre_signed_put_url | The pre-signed URL to be used for the PUT operation. | Yes | |
| jgroups.s3.prefix | If set, S3_PING searches for a bucket with a name that starts with the prefix value. | No |
30.2.2.4. default-jgroups-google.xml
The
default-configs/default-jgroups-google.xml file is a pre-configured JGroups configuration in Red Hat JBoss Data Grid. The default-jgroups-google.xml configuration
- uses TCP as a transport and GOOGLE_PING for discovery.
- is suitable on Google Compute Engine nodes where UDP multicast isn't available.
As with other pre-configured JGroups files, the behavior of some of these settings can be altered by adding certain system properties to the JVM at startup. The settings that can be configured are shown in the following table.
Table 30.4. default-jgroups-google.xml System Properties
| System Property | Description | Default | Required? |
|---|---|---|---|
| jgroups.tcp.address | IP address to use for the TCP transport. | 127.0.0.1 | No |
| jgroups.tcp.port | Port to use for TCP socket | 7800 | No |
| jgroups.google.access_key | The Google Compute Engine User's access key used to access the bucket | Yes | |
| jgroups.google.secret_access_key | The Google Compute Engine User's secret access key used to access the bucket | Yes | |
| jgroups.google.bucket | Name of the Google Compute Engine bucket to use. Must be unique and already exist | Yes |
Where did the comment section go?
Red Hat's documentation publication system recently went through an upgrade to enable speedier, more mobile-friendly content. We decided to re-evaluate our commenting platform to ensure that it meets your expectations and serves as an optimal feedback mechanism. During this redesign, we invite your input on providing feedback on Red Hat documentation via the discussion platform.