Chapter 11. The Hot Rod Interface

Hot Rod is a binary TCP client-server protocol used in Red Hat JBoss Data Grid. It was created to overcome deficiencies in other client/server protocols, such as Memcached.
Hot Rod will failover on a server cluster that undergoes a topology change. Hot Rod achieves this by providing regular updates to clients about the cluster topology.
Hot Rod enables clients to do smart routing of requests in partitioned or distributed JBoss Data Grid server clusters. To do this, Hot Rod allows clients to determine the partition that houses a key and then communicate directly with the server that has the key. This functionality relies on Hot Rod updating the cluster topology with clients, and that the clients use the same consistent hash algorithm as the servers.
JBoss Data Grid contains a server module that implements the Hot Rod protocol. The Hot Rod protocol facilitates faster client and server interactions in comparison to other text-based protocols and allows clients to make decisions about load balancing, failover and data location operations.

11.1. Hot Rod Headers

11.1.1. Hot Rod Header Data Types

All keys and values used for Hot Rod in Red Hat JBoss Data Grid are stored as byte arrays. Certain header values, such as those for REST and Memcached, are stored using the following data types instead:

Table 11.1. Header Data Types

Data Type Size Details
vInt Between 1-5 bytes. Unsigned variable length integer values.
vLong Between 1-9 bytes. Unsigned variable length long values.
string - Strings are always represented using UTF-8 encoding.

11.1.2. Request Header

When using Hot Rod to access Red Hat JBoss Data Grid, the contents of the request header consist of the following:

Table 11.2. Request Header Fields

Field Name Data Type/Size Details
Magic 1 byte Indicates whether the header is a request header or response header.
Message ID vLong Contains the message ID. Responses use this unique ID when responding to a request. This allows Hot Rod clients to implement the protocol in an asynchronous manner.
Version 1 byte Contains the Hot Rod server version.
Opcode 1 byte Contains the relevant operation code. In a request header, opcode can only contain the request operation codes.
Cache Name Length vInt Stores the length of the cache name. If Cache Name Length is set to 0 and no value is supplied for Cache Name, the operation interacts with the default cache.
Cache Name string Stores the name of the target cache for the specified operation. This name must match the name of a predefined cache in the cache configuration file.
Flags vInt Contains a numeric value of variable length that represents flags passed to the system. Each bit represents a flag, except the most significant bit, which is used to determine whether more bytes must be read. Using a bit to represent each flag facilitates the representation of flag combinations in a condensed manner.
Client Intelligence 1 byte Contains a value that indicates the client capabilities to the server.
Topology ID vInt Contains the last known view ID in the client. Basic clients supply the value 0 for this field. Clients that support topology or hash information supply the value 0 until the server responds with the current view ID, which is subsequently used until a new view ID is returned by the server to replace the current view ID.

11.1.3. Response Header

When using Hot Rod to access Red Hat JBoss Data Grid, the contents of the response header consist of the following:

Table 11.3. Response Header Fields

Field Name Data Type Details
Magic 1 byte Indicates whether the header is a request or response header.
Message ID vLong Contains the message ID. This unique ID is used to pair the response with the original request. This allows Hot Rod clients to implement the protocol in an asynchronous manner.
Opcode 1 byte Contains the relevant operation code. In a response header, opcode can only contain the response operation codes.
Status 1 byte Contains a code that represents the status of the response.
Topology Change Marker 1 byte Contains a marker byte that indicates whether the response is included in the topology change information.

11.1.4. Topology Change Headers

When using Hot Rod to access Red Hat JBoss Data Grid, response headers respond to changes in the cluster or view formation by looking for clients that can distinguish between different topologies or hash distributions. The Hot Rod server compares the current topology ID and the topology ID sent by the client and, if the two differ, it returns a new topology ID.

11.1.4.1. Topology Change Marker Values

The following is a list of valid values for the Topology Change Marker field in a response header:

Table 11.4. Topology Change Marker Field Values

Value Details
0 No topology change information is added.
1 Topology change information is added.

11.1.4.2. Topology Change Headers for Topology-Aware Clients

The response header sent to topology-aware clients when a topology change is returned by the server includes the following elements:

Table 11.5. Topology Change Header Fields

Response Header Fields Data Type/Size Details
Response Header with Topology Change Marker variable Refer to Section 11.1.3, “Response Header”.
Topology ID vInt Topology ID.
Num Servers in Topology vInt Contains the number of Hot Rod servers running in the cluster. This value can be a subset of the entire cluster if only some nodes are running Hot Rod servers.
mX: Host/IP Length vInt Contains the length of the hostname or IP address of an individual cluster member. Variable length allows this element to include hostnames, IPv4 and IPv6 addresses.
mX: Host/IP Address string Contains the hostname or IP address of an individual cluster member. The Hot Rod client uses this information to access the individual cluster member.
mX: Port Unsigned Short. 2 bytes Contains the port used by Hot Rod clients to communicate with the cluster member.
The three entries with the prefix mX, are repeated for each server in the topology. The first server in the topology's information fields will be prefixed with m1 and the numerical value is incremented by one for each additional server till the value of X equals the number of servers specified in the num servers in topology field.

11.1.4.3. Topology Change Headers for Hash Distribution-Aware Clients

The response header sent to clients when a topology change is returned by the server includes the following elements:

Table 11.6. Topology Change Header Fields

Field Data Type/Size Details
Response Header with Topology Change Marker variable Refer to Section 11.1.3, “Response Header”.
Topology ID vInt Topology ID.
Number Key Owners Unsigned short. 2 bytes. Contains the number of globally configured copies for each distributed key. Contains the value 0 if distribution is not configured on the cache.
Hash Function Version 1 byte Contains a pointer to the hash function in use. Contains the value 0 if distribution is not configured on the cache.
Hash Space Size vInt Contains the modulus used by JBoss Data Grid for all module arithmetic related to hash code generation. Clients use this information to apply the correct hash calculations to the keys. Contains the value 0 if distribution is not configured on the cache.
Number servers in topology vInt Contains the number of Hot Rod servers running in the cluster. This value can be a subset of the entire cluster if only some nodes are running Hot Rod servers. This value also represents the number of host to port pairings included in the header.
Number Virtual Nodes Owners vInt Contains the number of configured virtual nodes. Contains the value 0 if no virtual nodes are configured or if distribution is not configured on the cache.
mX: Host/IP Length vInt Contains the length of the hostname or IP address of an individual cluster member. Variable length allows this element to include hostnames, IPv4 and IPv6 addresses.
mX: Host/IP Address string Contains the hostname or IP address of an individual cluster member. The Hot Rod client uses this information to access the individual cluster member.
mX: Port Unsigned short. 2 bytes. Contains the port used by Hot Rod clients to communicate with the cluster member.
Hash Function Version 1 byte 0x03
Number of Segments in Topology vInt Total number of segments in the topology.
Number of Owners in Segment 1 byte This can be either 0, 1, or 2 owners.
First Wwner's Index vInt Given the list of all nodes, the position of this owner in this list. This is only present if number of owners for this segment is 1 or 2.
Second Owner's Index vInt Given the list of all nodes, the position of this owner in this list. This is only present if number of owners for this segment is 2.

Note

Even though it is possible to have more than 2 owners per segment, the Hot Rod protocol limits the number of owners to send for efficiency reasons.
The three entries with the prefix mX, are repeated for each server in the topology. The first server in the topology's information fields will be prefixed with m1 and the numerical value is incremented by one for each additional server till the value of X equals the number of servers specified in the num servers in topology field.