Menu Close

7.3. Protobuf Encoding

The Infinispan Query DSL can be used remotely via the Hot Rod client. In order to do this, protocol buffers are used to adopt a common format for storing cache entries and marshalling them.

7.3.1. Storing Protobuf Encoded Entities

Protobuf requires data to be structured. This is achieved by declaring Protocol Buffer message types in .proto files
For example:

Example 7.2. .library.proto

package book_sample;  
message Book {      
      required string title = 1;
      required string description = 2;
      required int32 publicationYear = 3; // no native Date type available in Protobuf
      
      repeated Author authors = 4;
}
message Author {
    required string name = 1;
    required string surname = 2;
}
The provided example:
  1. An entity named Book is placed in a package named book_sample.
    package book_sample;  
    message Book {
  2. The entity declares several fields of primitive types and a repeatable field named authors.
     required string title = 1;
          required string description = 2;
          required int32 publicationYear = 3; // no native Date type available in Protobuf
          
          repeated Author authors = 4;
    }
  3. The Author message instances are embedded in the Book message instance.
    message Author {
        required string name = 1;
        required string surname = 2;
    }

7.3.2. About Protobuf Messages

There are a few important things to note about Protobuf messages:
  • Nesting of messages is possible, however the resulting structure is strictly a tree, and never a graph.
  • There is no type inheritance.
  • Collections are not supported, however arrays can be easily emulated using repeated fields.

7.3.3. Using Protobuf with Hot Rod

Protobuf can be used with JBoss Data Grid's Hot Rod using the following two steps:
  1. Configure the client to use a dedicated marshaller, in this case, the ProtoStreamMarshaller. This marshaller uses the ProtoStream library to assist in encoding objects.

    Important

    In order to use the ProtoStreamMarshaller, the infinispan-remote-query-client Maven dependency must be added.
  2. Instruct ProtoStream library on how to marshall message types by registering per entity marshallers.

Example 7.3. Use the ProtoStreamMarshaller to Encode and Marshall Messages

import org.infinispan.client.hotrod.configuration.ConfigurationBuilder;
import org.infinispan.client.hotrod.marshall.ProtoStreamMarshaller;
import org.infinispan.protostream.SerializationContext;
ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
clientBuilder.addServer()
    .host("127.0.0.1").port(11234)
    .marshaller(new ProtoStreamMarshaller());
    
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build());
SerializationContext srcCtx = ProtoStreamMarshaller.getSerializationContext(remoteCacheManager);
serCtx.registerProtofile("/library.protobin");
serCtx.registerMarshaller(Book.class, new BookMarshaller());
serCtx.registerMarshaller(Author.class, new AuthorMarshaller());
// Book and Author classes omitted for brevity
In the provided example,
  • The SerializationContext is provided by the ProtoStream library.
  • The SerializationContext.registerProtofile method receives the name of a classpath resource that is a serialized protobuf binary descriptor containing the type declarations. The binary descriptor, .protobin, is compiled with Protobuf's protoc generator tool using the
    --descriptor_set_out
    command line option for the library.proto file.
  • The SerializationContext associated with the RemoteCacheManager is obtained, then ProtoStream is instructed to marshall the protobuf types.

Note

A RemoteCacheManager has no SerializationContext associated with it unless it was configured to use ProtoStreamMarshaller.

7.3.4. Registering Per Entity Marshallers

When using the ProtoStreamMarshaller for remote querying purposes, registration of per entity marshallers for domain model types must be provided by the user for each type or marshalling will fail. When writing marshallers, it is essential that they are stateless and threadsafe, as a single instance of them is being used.
The following example shows how to write a marshaller.

Example 7.4. BookMarshaller.java

import org.infinispan.protostream.MessageMarshaller;
...
public class BookMarshaller implements MessageMarshaller<Book> {
   @Override
   public String getTypeName() {
      return "book_sample.Book";
   }
   @Override
   public Class<? extends Book> getJavaClass() {
      return Book.class;
   }
   @Override
   public void writeTo(ProtoStreamWriter writer, Book book) throws IOException {
      writer.writeString("title", book.getTitle());
      writer.writeString("description", book.getDescription());
      writer.writeCollection("authors", book.getAuthors(), Author.class);
   }
   @Override
   public Book readFrom(ProtoStreamReader reader) throws IOException {
      String title = reader.readString("title");
      String description = reader.readString("description");
      int publicationYear = reader.readInt("publicationYear");
      Set<Author> authors = reader.readCollection("authors", new HashSet<Author>(), Author.class);
      return new Book(title, description, publicationYear, authors);
   }
}
Once the client has been set up, reading and writing Java objects to the remote cache. The actual data stored in the cache will be protobuf encoded, provided that marshallers were registered with the remote client for all involved types. In the provided example, this would be Book and Author.
Objects stored in protobuf format are able to be utilized with compatible clients written in different languages.

7.3.5. Indexing Protobuf Encoded Entities

Once the client has been configured to use Protobuf, indexing can be configured for caches on the server side.
To be able to index the entries, the server must extract relevant metadata from the same binary descriptor as the client, that is, the .protobin file. The descriptor is supplied to the server by remotely invoking the ProtobufMetadataManager MBean via JMX.
The ProtobufMetadataManager is a cluster-wide replicated repository of protobuf descriptors. For each running cache manager a separate ProtobufMetadataManager MBean instance exists. The ProtobufMetadataManager ObjectName uses the following pattern:
<jmx domain>:type=RemoteQuery,name=<cache manager 
		name>,component=ProtobufMetadataManager
The following signature is used by the method that registers the Protobuf descriptor file:
void registerProtofile(byte[] descriptorFile)
If indexing is enabled for a cache, all fields of Protobuf-encoded entries are indexed. All Protobuf-encoded entries are searchable, regardless of whether indexing is enabled.

Note

Indexing is recommended for improved performance but is not mandatory when using Remote Queries. Using indexing improves the searching speed but can also reduce the insert/update speeds due to the overhead required to maintain the index.