Chapter 27. Architectural Overview

This section contains a high level overview of Red Hat Data Grid’s internal architecture. This document is geared towards people with an interest in extending or enhancing Red Hat Data Grid, or just curious about Red Hat Data Grid’s internals.

27.1. Cache hierarchy

Red Hat Data Grid’s Cache interface extends the JRE’s ConcurrentMap interface which provides for a familiar and easy-to-use API.

---
public interface Cache<K, V> extends BasicCache<K, V> {
 ...
}

public interface BasicCache<K, V> extends ConcurrentMap<K, V> { …​ } ---

Caches are created by using a CacheContainer instance - either the EmbeddedCacheManager or a RemoteCacheManager. In addition to their capabilities as a factory for Caches, CacheContainers also act as a registry for looking up Caches.

EmbeddedCacheManagers create either clustered or standalone Caches that reside in the same JVM. RemoteCacheManagers, on the other hand, create RemoteCaches that connect to a remote cache tier via the Hot Rod protocol.

27.2. Commands

Internally, each and every cache operation is encapsulated by a command. These command objects represent the type of operation being performed, and also hold references to necessary parameters. The actual logic of a given command, for example a ReplaceCommand, is encapsulated in the command’s perform() method. Very object-oriented and easy to test.

All of these commands implement the VisitableCommand inteface which allow a Visitor (described in next section) to process them accordingly.

---
public class PutKeyValueCommand extends VisitableCommand {
  ...
}

public class GetKeyValueCommand extends VisitableCommand { …​ }

  1. etc …​ ---

27.3. Visitors

Commands are processed by the various Visitors. The visitor interface, displayed below, exposes methods to visit each of the different types of commands in the system. This gives us a type-safe mechanism for adding behaviour to a call.Commands are processed by `Visitor`s. The visitor interface, displayed below, exposes methods to visit each of the different types of commands in the system. This gives us a type-safe mechanism for adding behaviour to a call.

---
public interface Vistor {
   Object visitPutKeyValueCommand(InvocationContext ctx, PutKeyValueCommand command) throws Throwable;
Object visitRemoveCommand(InvocationContext ctx, RemoveCommand command) throws Throwable;
Object visitReplaceCommand(InvocationContext ctx, ReplaceCommand command) throws Throwable;
Object visitClearCommand(InvocationContext ctx, ClearCommand command) throws Throwable;
Object visitPutMapCommand(InvocationContext ctx, PutMapCommand command) throws Throwable;
  1. etc …​ } ---

An AbstractVisitor class in the org.infinispan.commands package is provided with no-op implementations of each of these methods. Real implementations then only need override the visitor methods for the commands that interest them, allowing for very concise, readable and testable visitor implementations.

27.4. Interceptors

Interceptors are special types of Visitors, which are capable of visiting commands, but also acts in a chain. A chain of interceptors all visit the command, one in turn, until all registered interceptors visit the command.

The class to note is the CommandInterceptor. This abstract class implements the interceptor pattern, and also implements Visitor. Red Hat Data Grid’s interceptors extend CommandInterceptor, and these add specific behaviour to specific commands, such as distribution across a network or writing through to disk.

There is also an experimental asynchronous interceptor which can be used. The interface used for asynchronous interceptors is AsyncInterceptor and a base implementation which should be used when a custom implementation is desired BaseCustomAsyncInterceptor. Note this class also implements the Visitor interface.

27.5. Putting it all together

So how does this all come together? Invocations on the cache cause the cache to first create an invocation context for the call. Invocation contexts contain, among other things, transactional characteristics of the call. The cache then creates a command for the call, making use of a command factory which initialises the command instance with parameters and references to other subsystems.

The cache then passes the invocation context and command to the InterceptorChain, which calls each and every registered interceptor in turn to visit the command, adding behaviour to the call. Finally, the command’s perform() method is invoked and the return value, if any, is propagated back to the caller.

27.6. Subsystem Managers

The interceptors act as simple interception points and don’t contain a lot of logic themselves. Most behavioural logic is encapsulated as managers in various subsystems, a small subset of which are:

27.6.1. DistributionManager

Manager that controls how entries are distributed across the cluster.

27.6.2. TransactionManager

Manager than handles transactions, usually supplied by a third party.

27.6.3. RpcManager

Manager that handles replicating commands between nodes in the cluster.

27.6.4. LockManager

Manager that handles locking keys when operations require them.

27.6.5. PersistenceManager

Manager that handles persisting data to any configured cache stores.

27.6.6. DataContainer

Container that holds the actual in memory entries.

27.6.7. Configuration

A component detailing all of the configuration in this cache.

27.7. ComponentRegistry

A registry where the various managers above and components are created and stored for use in the cache. All of the other managers and crucial componanets are accesible through the registry.

The registry itself is a lightweight dependency injection framework, allowing components and managers to reference and initialise one another. Here is an example of a component declaring a dependency on a DataContainer and a Configuration, and a DataContainerFactory declaring its ability to construct DataContainers on the fly.

---
   @Inject
   public void injectDependencies(DataContainer container, Configuration configuration) {
      this.container = container;
      this.configuration = configuration;
   }
   @DefaultFactoryFor
   public class DataContainerFactory extends AbstractNamedCacheComponentFactory {
---

Components registered with the ComponentRegistry may also have a lifecycle, and methods annotated with @Start or @Stop will be invoked before and after they are used by the component registry.

---
   @Start
   public void init() {
      useWriteSkewCheck = configuration.locking().writeSkewCheck();
   }
   @Stop(priority=20)
   public void stop() {
      notifier.removeListener(listener);
      executor.shutdownNow();
   }
---

In the example above, the optional priority parameter to @Stop is used to indicate the order in which the component is stopped, in relation to other components. This follows a Unix Sys-V style ordering, where smaller priority methods are called before higher priority ones. The default priority, if not specified, is 10.