Chapter 6. Querying Caches with Protobuf Metadata

Data Grid supports using Protocol Buffers (Protobuf) to structure data in the cache so that you can query it.

Prerequisites

  • Start the Data Grid CLI.
  • Connect to a running Data Grid cluster.

6.1. Configuring Media Types

Encode cache entries with different media types to store data in a format that best suits your requirements.

For example, the following procedure shows you how to configure the application/x-protostream media type.

Procedure

  1. Create a Data Grid configuration file that adds a distributed cache named qcache and configures the media type, for example: 

    <infinispan>
   <cache-container>
      <distributed-cache name="qcache">
         <encoding>
            <key media-type="application/x-protostream"/>
            <value media-type="application/x-protostream"/>
         </encoding>
      </distributed-cache>
   </cache-container>
</infinispan>

  2. Create qcache from pcache.xml with the --file= option. 

    [//containers/default]> create cache --file=pcache.xml pcache

  3. Verify pcache. 

    [//containers/default]> ls caches
pcache
___protobuf_metadata
[//containers/default]> describe caches/pcache
{
  "distributed-cache" : {
    "mode" : "SYNC",
    "encoding" : {
      "key" : {
        "media-type" : "application/x-protostream"
      },
      "value" : {
        "media-type" : "application/x-protostream"
      }
    },
    "transaction" : {
      "mode" : "NONE"
    }
  }
}

  4. Add an entry to pcache and check the encoding. 

    [//containers/default]> put --cache=pcache good morning
[//containers/default]> cd caches/pcache
[//containers/default/caches/pcache]> get good
{
  "_type" : "string",
  "_value" : "morning"
}

6.2. Registering Protobuf Schemas

Protobuf schemas contain data structures known as messages in .proto definition files.

Procedure

  1. Create a schema file named person.proto with the following messages: 

    package org.infinispan.rest.search.entity;

message Address {
    required string street = 1;
    required string postCode = 2;
}

message PhoneNumber {
    required string number = 1;
}

message Person {
    optional int32 id = 1;
    required string name = 2;
    required string surname = 3;
    optional Address address = 4;
    repeated PhoneNumber phoneNumbers = 5;
    optional uint32 age = 6;
    enum Gender {
        MALE = 0;
        FEMALE = 1;
    }

    optional Gender gender = 7;
}

  2. Register person.proto. 

    [//containers/default]> schema --upload=person.proto person.proto

  3. Verify person.proto. 

    [//containers/default]> cd caches/___protobuf_metadata
[//containers/default/caches/___protobuf_metadata]> ls
person.proto
[//containers/default/caches/___protobuf_metadata]> get person.proto

6.3. Querying Caches with Protobuf Schemas

Data Grid automatically converts JSON to Protobuf so that you can read and write cache entries in JSON format and use Protobuf schemas to query them.

For example, consider the following JSON documents:

lukecage.json

{
  "_type":"org.infinispan.rest.search.entity.Person",
  "id":2,
  "name":"Luke",
  "surname":"Cage",
  "gender":"MALE",
  "address":{"street":"38th St","postCode":"NY 11221"},
  "phoneNumbers":[{"number":4444},{"number":5555}]
}

jessicajones.json

{
  "_type":"org.infinispan.rest.search.entity.Person",
  "id":1,
  "name":"Jessica",
  "surname":"Jones",
  "gender":"FEMALE",
  "address":{"street":"46th St","postCode":"NY 10036"},
  "phoneNumbers":[{"number":1111},{"number":2222},{"number":3333}]
}

matthewmurdock.json

{
  "_type":"org.infinispan.rest.search.entity.Person",
  "id":3,
  "name":"Matthew",
  "surname":"Murdock",
  "gender":"MALE",
  "address":{"street":"57th St","postCode":"NY 10019"},
  "phoneNumbers":[]
}

Each of the preceding JSON documents contains:

  • a _type field that identifies the Protobuf message to which the JSON document corresponds.
  • several fields that correspond to datatypes in the person.proto schema.

Procedure

  1. Navigate to the pcache cache. 

    [//containers/default/caches]> cd pcache

  2. Add each JSON document as an entry to the cache, for example: 

    [//containers/default/caches/pcache]> put --encoding=application/json --file=jessicajones.json jessicajones
[//containers/default/caches/pcache]> put --encoding=application/json --file=matthewmurdock.json matthewmurdock
[//containers/default/caches/pcache]> put --encoding=application/json --file=lukecage.json lukecage

  3. Verify that the entries exist. 

    [//containers/default/caches/pcache]> ls
lukecage
matthewmurdock
jessicajones

  4. Query the cache to return entries from the Protobuf Person entity where the gender datatype is MALE. 

    [//containers/default/caches/pcache]> query "from org.infinispan.rest.search.entity.Person p where p.gender = 'MALE'"
{
  "total_results" : 2,
  "hits" : [ {
    "hit" : {
      "_type" : "org.infinispan.rest.search.entity.Person",
      "id" : 2,
      "name" : "Luke",
      "surname" : "Cage",
      "gender" : "MALE",
      "address" : {
        "street" : "38th St",
        "postCode" : "NY 11221"
      },
      "phoneNumbers" : [ {
        "number" : "4444"
      }, {
        "number" : "5555"
      } ]
    }
  }, {
    "hit" : {
      "_type" : "org.infinispan.rest.search.entity.Person",
      "id" : 3,
      "name" : "Matthew",
      "surname" : "Murdock",
      "gender" : "MALE",
      "address" : {
        "street" : "57th St",
        "postCode" : "NY 10019"
      }
    }
  } ]
}