Chapter 3. Hot Rod Java Client Configuration

Data Grid provides a Hot Rod Java client configuration API that exposes configuration properties.

3.1. Adding Hot Rod Java Client Dependencies

Add Hot Rod Java client dependencies to include it in your project.

Prerequisites

  • Java 8 or Java 11

Procedure

  • Add the infinispan-client-hotrod artifact as a dependency in your pom.xml as follows:
<dependency>
  <groupId>org.infinispan</groupId>
  <artifactId>infinispan-client-hotrod</artifactId>
</dependency>

3.2. Configuring Hot Rod Client Connections

Configure Hot Rod Java client connections to Data Grid Server.

Procedure

  • Use the ConfigurationBuilder class to generate immutable configuration objects that you can pass to RemoteCacheManager or use a hotrod-client.properties file on the application classpath.

ConfigurationBuilder

ConfigurationBuilder builder = new ConfigurationBuilder();
builder.addServer()
         .host("127.0.0.1")
         .port(ConfigurationProperties.DEFAULT_HOTROD_PORT)
       .addServer()
         .host("192.0.2.0")
         .port(ConfigurationProperties.DEFAULT_HOTROD_PORT)
       .security().authentication()
         .username("username")
         .password("changeme")
         .realm("default")
         .saslMechanism("SCRAM-SHA-512");
RemoteCacheManager cacheManager = new RemoteCacheManager(builder.build());

hotrod-client.properties

infinispan.client.hotrod.server_list = 127.0.0.1:11222,192.0.2.0:11222
infinispan.client.hotrod.auth_username = username
infinispan.client.hotrod.auth_password = changeme
infinispan.client.hotrod.auth_realm = default
infinispan.client.hotrod.sasl_mechanism = SCRAM-SHA-512

Configuring Hot Rod URIs

You can also configure Hot Rod client connections with URIs as follows:

ConfigurationBuilder

ConfigurationBuilder builder = new ConfigurationBuilder();
builder.uri("hotrod://username:changeme@127.0.0.1:11222,192.0.2.0:11222?auth_realm=default&sasl_mechanism=SCRAM-SHA-512");
RemoteCacheManager cacheManager = new RemoteCacheManager(builder.build());

hotrod-client.properties

infinispan.client.hotrod.uri = hotrod://username:changeme@127.0.0.1:11222,192.0.2.0:11222?auth_realm=default&sasl_mechanism=SCRAM-SHA-512

Adding properties outside the classpath

If the hotrod-client.properties file is not on the application classpath then you need to specify the location, as in the following example:

ConfigurationBuilder builder = new ConfigurationBuilder();
Properties p = new Properties();
try(Reader r = new FileReader("/path/to/hotrod-client.properties")) {
   p.load(r);
   builder.withProperties(p);
}
RemoteCacheManager cacheManager = new RemoteCacheManager(builder.build());

3.2.1. Defining Data Grid Clusters in Client Configuration

Provide the locations of Data Grid clusters in Hot Rod client configuration.

Procedure

  • Provide at least one Data Grid cluster name along with a host name and port for at least one node with the ClusterConfigurationBuilder class.

    If you want to define a cluster as default, so that clients always attempt to connect to it first, then define a server list with the addServers("<host_name>:<port>; <host_name>:<port>") method.

Multiple cluster connections

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addCluster("siteA")
               .addClusterNode("hostA1", 11222)
               .addClusterNode("hostA2", 11222)
             .addCluster("siteB")
               .addClusterNodes("hostB1:11222; hostB2:11222");
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build());

Default server list with a failover cluster

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServers("hostA1:11222; hostA2:11222")
             .addCluster("siteB")
               .addClusterNodes("hostB1:11222; hostB2:11223");
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build());

3.2.2. Manually Switching Data Grid Clusters

Manually switch Hot Rod Java client connections between Data Grid clusters.

Procedure

  • Call one of the following methods in the RemoteCacheManager class:

    switchToCluster(clusterName) switches to a specific cluster defined in the client configuration.

    switchToDefaultCluster() switches to the default cluster in the client configuration, which is defined as a list of Data Grid servers.

Additional resources

3.2.3. Configuring Connection Pools

Hot Rod Java clients keep pools of persistent connections to Data Grid servers to reuse TCP connections instead of creating them on each request.

Procedure

  • Configure Hot Rod client connection pool settings as in the following examples:

ConfigurationBuilder

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .connectionPool()
               .maxActive(10)
               exhaustedAction(ExhaustedAction.valueOf("WAIT"))
               .maxWait(1)
               .minIdle(20)
               .minEvictableIdleTime(300000)
               .maxPendingRequests(20);
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build());

hotrod-client.properties

infinispan.client.hotrod.server_list = 127.0.0.1:11222
infinispan.client.hotrod.connection_pool.max_active = 10
infinispan.client.hotrod.connection_pool.exhausted_action = WAIT
infinispan.client.hotrod.connection_pool.max_wait = 1
infinispan.client.hotrod.connection_pool.min_idle = 20
infinispan.client.hotrod.connection_pool.min_evictable_idle_time = 300000
infinispan.client.hotrod.connection_pool.max_pending_requests = 20

3.3. Configuring Authentication Mechanisms for Hot Rod Clients

Data Grid Server uses different mechanisms to authenticate Hot Rod client connections.

Procedure

  • Specify authentication mechanisms with the saslMechanism() method from the AuthenticationConfigurationBuilder class or with the infinispan.client.hotrod.sasl_mechanism property.

SCRAM

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security()
               .authentication()
                 .saslMechanism("SCRAM-SHA-512")
                 .username("myuser")
                 .password("qwer1234!");

DIGEST

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security()
               .authentication()
                 .saslMechanism("DIGEST-MD5")
                 .username("myuser")
                 .password("qwer1234!");

PLAIN

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security()
               .authentication()
                 .saslMechanism("PLAIN")
                 .username("myuser")
                 .password("qwer1234!");

OAUTHBEARER

String token = "..."; // Obtain the token from your OAuth2 provider
ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security()
               .authentication()
                 .saslMechanism("OAUTHBEARER")
                 .token(token);

EXTERNAL

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder
   .addServer()
      .host("127.0.0.1")
      .port(11222)
   .security()
      .ssl()
         // TrustStore stores trusted CA certificates for the server.
         .trustStoreFileName("/path/to/truststore")
         .trustStorePassword("truststorepassword".toCharArray())
         .trustStoreType("PCKS12")
         // KeyStore stores valid client certificates.
         .keyStoreFileName("/path/to/keystore")
         .keyStorePassword("keystorepassword".toCharArray())
         .keyStoreType("PCKS12")
      .authentication()
         .saslMechanism("EXTERNAL");
remoteCacheManager = new RemoteCacheManager(clientBuilder.build());
RemoteCache<String, String> cache = remoteCacheManager.getCache("secured");

GSSAPI

LoginContext lc = new LoginContext("GssExample", new BasicCallbackHandler("krb_user", "krb_password".toCharArray()));
lc.login();
Subject clientSubject = lc.getSubject();

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security()
               .authentication()
                 .saslMechanism("GSSAPI")
                 .clientSubject(clientSubject)
                 .callbackHandler(new BasicCallbackHandler());

Basic Callback Handler

The BasicCallbackHandler, as shown in the GSSAPI example, invokes the following callbacks:

  • NameCallback and PasswordCallback construct the client subject.
  • AuthorizeCallback is called during SASL authentication.

OAUTHBEARER with Token Callback Handler

Use a TokenCallbackHandler to refresh OAuth2 tokens before they expire, as in the following example:

String token = "..."; // Obtain the token from your OAuth2 provider
TokenCallbackHandler tokenHandler = new TokenCallbackHandler(token);
ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security()
               .authentication()
               .saslMechanism("OAUTHBEARER")
               .callbackHandler(tokenHandler);
remoteCacheManager = new RemoteCacheManager(clientBuilder.build());
RemoteCache<String, String> cache = remoteCacheManager.getCache("secured");
// Refresh the token
tokenHandler.setToken("newToken");

Custom CallbackHandler

Hot Rod clients set up a default CallbackHandler to pass credentials to SASL mechanisms. In some cases you might need to provide a custom CallbackHandler, as in the following example:

public class MyCallbackHandler implements CallbackHandler {
   final private String username;
   final private char[] password;
   final private String realm;

   public MyCallbackHandler(String username, String realm, char[] password) {
      this.username = username;
      this.password = password;
      this.realm = realm;
   }

   @Override
   public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
      for (Callback callback : callbacks) {
         if (callback instanceof NameCallback) {
            NameCallback nameCallback = (NameCallback) callback;
            nameCallback.setName(username);
         } else if (callback instanceof PasswordCallback) {
            PasswordCallback passwordCallback = (PasswordCallback) callback;
            passwordCallback.setPassword(password);
         } else if (callback instanceof AuthorizeCallback) {
            AuthorizeCallback authorizeCallback = (AuthorizeCallback) callback;
            authorizeCallback.setAuthorized(authorizeCallback.getAuthenticationID().equals(
                  authorizeCallback.getAuthorizationID()));
         } else if (callback instanceof RealmCallback) {
            RealmCallback realmCallback = (RealmCallback) callback;
            realmCallback.setText(realm);
         } else {
            throw new UnsupportedCallbackException(callback);
         }
      }
   }
}

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
               .host("127.0.0.1")
               .port(11222)
             .security().authentication()
               .serverName("myhotrodserver")
               .saslMechanism("DIGEST-MD5")
               .callbackHandler(new MyCallbackHandler("myuser","default","qwer1234!".toCharArray()));
Note

A custom CallbackHandler needs to handle callbacks that are specific to the authentication mechanism that you use. However, it is beyond the scope of this document to provide examples for each possible callback type.

3.3.1. Creating GSSAPI Login Contexts

To use the GSSAPI mechanism, you must create a LoginContext so your Hot Rod client can obtain a Ticket Granting Ticket (TGT).

Procedure

  1. Define a login module in a login configuration file.

    gss.conf

    GssExample {
        com.sun.security.auth.module.Krb5LoginModule required client=TRUE;
    };

    For the IBM JDK:

    gss-ibm.conf

    GssExample {
        com.ibm.security.auth.module.Krb5LoginModule required client=TRUE;
    };

  2. Set the following system properties:

    java.security.auth.login.config=gss.conf
    
    java.security.krb5.conf=/etc/krb5.conf
    Note

    krb5.conf provides the location of your KDC. Use the kinit command to authenticate with Kerberos and verify krb5.conf.

3.3.2. SASL authentication mechanisms

Data Grid Server supports the following SASL authentications mechanisms with Hot Rod endpoints:

Authentication mechanismDescriptionSecurity realm typeRelated details

PLAIN

Uses credentials in plain-text format. You should use PLAIN authentication with encrypted connections only.

Property realms and LDAP realms

Similar to the BASIC HTTP mechanism.

DIGEST-*

Uses hashing algorithms and nonce values. Hot Rod connectors support DIGEST-MD5, DIGEST-SHA, DIGEST-SHA-256, DIGEST-SHA-384, and DIGEST-SHA-512 hashing algorithms, in order of strength.

Property realms and LDAP realms

Similar to the Digest HTTP mechanism.

SCRAM-*

Uses salt values in addition to hashing algorithms and nonce values. Hot Rod connectors support SCRAM-SHA, SCRAM-SHA-256, SCRAM-SHA-384, and SCRAM-SHA-512 hashing algorithms, in order of strength.

Property realms and LDAP realms

Similar to the Digest HTTP mechanism.

GSSAPI

Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding kerberos server identity in the realm configuration. In most cases, you also specify an ldap-realm to provide user membership information.

Kerberos realms

Similar to the SPNEGO HTTP mechanism.

GS2-KRB5

Uses Kerberos tickets and requires a Kerberos Domain Controller. You must add a corresponding kerberos server identity in the realm configuration. In most cases, you also specify an ldap-realm to provide user membership information.

Kerberos realms

Similar to the SPNEGO HTTP mechanism.

EXTERNAL

Uses client certificates.

Trust store realms

Similar to the CLIENT_CERT HTTP mechanism.

OAUTHBEARER

Uses OAuth tokens and requires a token-realm configuration.

Token realms

Similar to the BEARER_TOKEN HTTP mechanism.

3.4. Configuring Hot Rod client encryption

Data Grid Server can enforce SSL/TLS encryption and present Hot Rod clients with certificates to establish trust and negotiate secure connections.

To verify certificates issued to Data Grid Server, Hot Rod clients require either the full certificate chain or a partial chain that starts with the Root CA. You provide server certificates to Hot Rod clients as trust stores.

Tip

Alternatively to providing trust stores you can use shared system certificates.

Prerequisites

  • Create a trust store that Hot Rod clients can use to verify Data Grid Server identities.
  • If you configure Data Grid Server to validate or authenticate client certificates, create a keystore as appropriate.

Procedure

  1. Add the trust store to the client configuration with the trustStoreFileName() and trustStorePassword() methods or corresponding properties.
  2. If you configure client certificate authentication, do the following:

    1. Add the keystore to the client configuration with the keyStoreFileName() and keyStorePassword() methods or corresponding properties.
    2. Configure clients to use the EXTERNAL authentication mechanism.

ConfigurationBuilder

ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder
   .addServer()
      .host("127.0.0.1")
      .port(11222)
   .security()
      .ssl()
         // Server SNI hostname.
         .sniHostName("myservername")
         // Keystore that contains the public keys for Data Grid Server.
         // Clients use the trust store to verify Data Grid Server identities.
         .trustStoreFileName("/path/to/server/truststore")
         .trustStorePassword("truststorepassword".toCharArray())
         .trustStoreType("PCKS12")
         // Keystore that contains client certificates.
         // Clients present these certificates to Data Grid Server.
         .keyStoreFileName("/path/to/client/keystore")
         .keyStorePassword("keystorepassword".toCharArray())
         .keyStoreType("PCKS12")
      .authentication()
         // Clients must use the EXTERNAL mechanism for certificate authentication.
         .saslMechanism("EXTERNAL");

hotrod-client.properties

infinispan.client.hotrod.server_list = 127.0.0.1:11222
infinispan.client.hotrod.use_ssl = true
infinispan.client.hotrod.sni_host_name = myservername
# Keystore that contains the public keys for Data Grid Server.
# Clients use the trust store to verify Data Grid Server identities.
infinispan.client.hotrod.trust_store_file_name = server_truststore.pkcs12
infinispan.client.hotrod.trust_store_password = changeme
infinispan.client.hotrod.trust_store_type = PCKS12
# Keystore that contains client certificates.
# Clients present these certificates to Data Grid Server.
infinispan.client.hotrod.key_store_file_name = client_keystore.pkcs12
infinispan.client.hotrod.key_store_password = changeme
infinispan.client.hotrod.key_store_type = PCKS12
# Clients must use the EXTERNAL mechanism for certificate authentication.
infinispan.client.hotrod.sasl_mechanism = EXTERNAL

Next steps

Add a client trust store to the $RHDG_HOME/server/conf directory and configure Data Grid Server to use it, if necessary.

3.5. Enabling Hot Rod client statistics

Hot Rod Java clients can provide statistics that include remote cache and near-cache hits and misses as well as connection pool usage.

Procedure

  1. Open your Hot Rod Java client configuration for editing.
  2. Set true as the value for the statistics property or invoke the statistics().enable() methods.
  3. Export JMX MBeans for your Hot Rod client with the jmx and jmx_domain properties or invoke the jmxEnable() and jmxDomain() methods.
  4. Save and close your client configuration.

Hot Rod Java client statistics

ConfigurationBuilder

ConfigurationBuilder builder = new ConfigurationBuilder();
builder.statistics().enable()
         .jmxEnable()
         .jmxDomain("my.domain.org")
       .addServer()
         .host("127.0.0.1")
         .port(11222);
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(builder.build());

hotrod-client.properties

infinispan.client.hotrod.statistics = true
infinispan.client.hotrod.jmx = true
infinispan.client.hotrod.jmx_domain = my.domain.org

3.6. Near Caches

Near caches are local to Hot Rod clients and store recently used data so that every read operation does not need to traverse the network, which significantly increases performance.

Near caches:

  • Are populated with read operations, calls to get() or getVersioned() methods.
    In the following example the put() call does not populate the near cache and only has the effect of invalidating the entry if it already exists:

    cache.put("k1", "v1");
    cache.get("k1");
  • Register a client listener to invalidate entries when they are updated or removed in remote caches on Data Grid Server.
    If entries are requested after they are invalidated, clients must retrieve them from the remote caches again.
  • Are cleared when clients fail over to different servers.

Bounded near caches

You should always use bounded near caches by specifying the maximum number of entries they can contain. When near caches reach the maximum number of entries, eviction automatically takes place to remove older entries. This means you do not need to manually keep the cache size within the boundaries of the client JVM.

Important

Do not use maximum idle expiration with near caches because near-cache reads do not propagate the last access time for entries.

Bloom filters

Bloom filters optimize performance for write operations by reducing the total number of invalidation messages.

Bloom filters:

  • Reside on Data Grid Server and keep track of the entries that the client has requested.
  • Require a connection pool configuration that has a maximum of one active connection per server and uses the WAIT exhausted action.
  • Cannot be used with unbounded near caches.

3.6.1. Configuring Near Caches

Configure Hot Rod Java clients with near caches to store recently used data locally in the client JVM.

Procedure

  1. Open your Hot Rod Java client configuration.
  2. Configure each cache to perform near caching with the nearCacheMode(NearCacheMode.INVALIDATED) method.

    Note

    Data Grid provides global near cache configuration properties. However, those properties are deprecated and you should not use them but configure near caching on a per-cache basis instead.

  3. Specify the maximum number of entries that the near cache can hold before eviction occurs with the nearCacheMaxEntries() method.
  4. Enable bloom filters for near caches with the nearCacheUseBloomFilter() method.
import org.infinispan.client.hotrod.configuration.ConfigurationBuilder;
import org.infinispan.client.hotrod.configuration.NearCacheMode;
import org.infinispan.client.hotrod.configuration.ExhaustedAction;

ConfigurationBuilder builder = new ConfigurationBuilder();
builder.addServer()
    .host("127.0.0.1")
    .port(ConfigurationProperties.DEFAULT_HOTROD_PORT)
  .security().authentication()
    .username("username")
    .password("password")
    .realm("default")
    .saslMechanism("SCRAM-SHA-512")
  // Configure the connection pool for bloom filters.
  .connectionPool()
    .maxActive(1)
    .exhaustedAction(ExhaustedAction.WAIT);
// Configure near caching for specific caches
builder.remoteCache("books")
    .nearCacheMode(NearCacheMode.INVALIDATED)
    .nearCacheMaxEntries(100)
    .nearCacheUseBloomFilter(false);
builder.remoteCache("authors")
    .nearCacheMode(NearCacheMode.INVALIDATED)
    .nearCacheMaxEntries(200)
    .nearCacheUseBloomFilter(true);

3.7. Forcing Return Values

To avoid sending data unnecessarily, write operations on remote caches return null instead of previous values.

For example, the following method calls do not return previous values for keys:

V remove(Object key);
V put(K key, V value);

You can, however, change the default behavior so your invocations return previous values for keys.

Procedure

  • Configure Hot Rod clients so method calls return previous values for keys in one of the following ways:

FORCE_RETURN_VALUE flag

cache.withFlags(Flag.FORCE_RETURN_VALUE).put("aKey", "newValue")

Per-cache

ConfigurationBuilder builder = new ConfigurationBuilder();
// Return previous values for keys for invocations for a specific cache.
builder.remoteCache("mycache")
       .forceReturnValues(true);

hotrod-client.properties

# Use the "*" wildcard in the cache name to return previous values
# for all caches that start with the "somecaches" string.

infinispan.client.hotrod.cache.somecaches*.force_return_values = true

Additional resources

3.8. Creating remote caches from Hot Rod clients

Use the Data Grid Hot Rod API to create remote caches on Data Grid Server from Java, C++, .NET/C#, JS clients and more.

This procedure shows you how to use Hot Rod Java clients that create remote caches on first access. You can find code examples for other Hot Rod clients in the Data Grid Tutorials.

Prerequisites

  • Create a Data Grid user with admin permissions.
  • Start at least one Data Grid Server instance.
  • Have a Data Grid cache configuration.

Procedure

  • Invoke the remoteCache() method as part of your the ConfigurationBuilder.
  • Set the configuration or configuration_uri properties in the hotrod-client.properties file on your classpath.

ConfigurationBuilder

File file = new File("path/to/infinispan.xml")
ConfigurationBuilder builder = new ConfigurationBuilder();
builder.remoteCache("another-cache")
       .configuration("<distributed-cache name=\"another-cache\"/>");
builder.remoteCache("my.other.cache")
       .configurationURI(file.toURI());

hotrod-client.properties

infinispan.client.hotrod.cache.another-cache.configuration=<distributed-cache name=\"another-cache\"/>
infinispan.client.hotrod.cache.[my.other.cache].configuration_uri=file:///path/to/infinispan.xml

Important

If the name of your remote cache contains the . character, you must enclose it in square brackets when using hotrod-client.properties files.