Upgrading Data Grid
Upgrade Data Grid to 8.1
Abstract
Red Hat Data Grid
Data Grid is a high-performance, distributed in-memory data store.
- Schemaless data structure
- Flexibility to store different objects as key-value pairs.
- Grid-based data storage
- Designed to distribute and replicate data across clusters.
- Elastic scaling
- Dynamically adjust the number of nodes to meet demand without service disruption.
- Data interoperability
- Store, retrieve, and query data in the grid from different endpoints.
Data Grid documentation
Documentation for Data Grid is available on the Red Hat customer portal.
Data Grid downloads
Access the Data Grid Software Downloads on the Red Hat customer portal.
You must have a Red Hat account to access and download Data Grid software.
Making open source more inclusive
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
Chapter 1. Performing Rolling Upgrades for Data Grid Servers
Perform rolling upgrades of your Data Grid clusters to change between versions without downtime or data loss. Rolling upgrades migrate both your Data Grid servers and your data to the target version over Hot Rod.
1.1. Setting Up Target Clusters
Create a cluster that runs the target Data Grid version and uses a remote cache store to load data from the source cluster.
Prerequisites
- Install a Data Grid cluster with the target upgrade version.
Ensure the network properties for the target cluster do not overlap with those for the source cluster. You should specify unique names for the target and source clusters in the JGroups transport configuration. Depending on your environment you can also use different network interfaces and specify port offsets to keep the target and source clusters separate.
Procedure
Add a
RemoteCacheStore
on the target cluster for each cache you want to migrate from the source cluster.Remote cache stores use the Hot Rod protocol to retrieve data from remote Data Grid clusters. When you add the remote cache store to the target cluster, it can lazily load data from the source cluster to handle client requests.
Switch clients over to the target cluster so it starts handling all requests.
- Update client configuration with the location of the target cluster.
- Restart clients.
1.1.1. Remote Cache Stores for Rolling Upgrades
You must use specific remote cache store configuration to perform rolling upgrades, as follows:
<persistence passivation="false"> 1 <remote-store xmlns="urn:infinispan:config:store:remote:11.0" cache="myDistCache" 2 protocol-version="2.5" 3 hotrod-wrapping="true" 4 raw-values="true" 5 segmented="false"> 6 <remote-server host="127.0.0.1" port="11222"/> 7 </remote-store> </persistence>
- 1
- Disables passivation. Remote cache stores for rolling upgrades must disable passivation.
- 2
- Matches the name of a cache in the source cluster. Target clusters load data from this cache using the remote cache store.
- 3
- Matches the Hot Rod protocol version of the source cluster.
2.5
is the minimum version and is suitable for any upgrade paths. You do not need to set another Hot Rod version. - 4
- Ensures that entries are wrapped in a suitable format for the Hot Rod protocol.
- 5
- Stores data in the remote cache store in raw format. This ensures that clients can use data directly from the remote cache store.
- 6
- Disables segmentation for the remote cache store. You should enable segmentation for remote cache stores only if the number of segments in the target cluster matches the number of segements for the cache in the source cluster.
- 7
- Points to the location of the source cluster.
1.2. Synchronizing Data to Target Clusters
When your target cluster is running and handling client requests using a remote cache store to load data on demand, you can synchronize data from the source cluster to the target cluster.
This operation reads data from the source cluster and writes it to the target cluster. Data migrates to all nodes in the target cluster in parallel, with each node receiving a subset of the data. You must perform the synchronization for each cache in your Data Grid configuration.
Procedure
Start the synchronization operation for each cache in your Data Grid configuration that you want to migrate to the target cluster.
Use the Data Grid REST API and invoke
POST
requests with the?action=sync- data
parameter. For example, to synchronize data in a cache named "myCache" from a source cluster to a target cluster, do the following:POST /v2/caches/myCache?action=sync-data
When the operation completes, Data Grid responds with the total number of entries copied to the target cluster.
Alternatively, you can use JMX by invoking
synchronizeData(migratorName=hotrod)
on theRollingUpgradeManager
MBean.Disconnect each node in the target cluster from the source cluster.
For example, to disconnect the "myCache" cache from the source cluster, invoke the following
POST
request:POST /v2/caches/myCache?action=disconnect-source
To use JMX, invoke
disconnectSource(migratorName=hotrod)
on theRollingUpgradeManager
MBean.
Next steps
After you synchronize all data from the source cluster, the rolling upgrade process is complete. You can now decommission the source cluster.
Chapter 2. Migrating Data Between Cache Stores
Data Grid provides a Java utility for migrating persisted data between cache stores.
In the case of upgrading Data Grid, functional differences between major versions do not allow backwards compatibility between cache stores. You can use StoreMigrator
to convert your data so that it is compatible with the target version.
For example, upgrading to Data Grid 8.0 changes the default marshaller to Protostream. In previous Data Grid versions, cache stores use a binary format that is not compatible with the changes to marshalling. This means that Data Grid 8.0 cannot read from cache stores with previous Data Grid versions.
In other cases Data Grid versions deprecate or remove cache store implementations, such as JDBC Mixed and Binary stores. You can use StoreMigrator
in these cases to convert to different cache store implementations.
2.1. Cache Store Migrator
Data Grid provides the StoreMigrator.java
utility that recreates data for the latest Data Grid cache store implementations.
StoreMigrator
takes a cache store from a previous version of Data Grid as source and uses a cache store implementation as target.
When you run StoreMigrator
, it creates the target cache with the cache store type that you define using the EmbeddedCacheManager
interface. StoreMigrator
then loads entries from the source store into memory and then puts them into the target cache.
StoreMigrator
also lets you migrate data from one type of cache store to another. For example, you can migrate from a JDBC String-Based cache store to a Single File cache store.
StoreMigrator
cannot migrate data from segmented cache stores to:
- Non-segmented cache store.
- Segmented cache stores that have a different number of segments.
2.2. Getting the Store Migrator
StoreMigrator
is available as part of the Data Grid tools library, infinispan-tools
, and is included in the Maven repository.
Procedure
Configure your
pom.xml
forStoreMigrator
as follows:<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.infinispan.example</groupId> <artifactId>jdbc-migrator-example</artifactId> <version>1.0-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.infinispan</groupId> <artifactId>infinispan-tools</artifactId> </dependency> <!-- Additional dependencies --> </dependencies> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.2.1</version> <executions> <execution> <goals> <goal>java</goal> </goals> </execution> </executions> <configuration> <mainClass>org.infinispan.tools.store.migrator.StoreMigrator</mainClass> <arguments> <argument>path/to/migrator.properties</argument> </arguments> </configuration> </plugin> </plugins> </build> </project>
2.3. Configuring the Store Migrator
Set properties for source and target cache stores in a migrator.properties
file.
Procedure
-
Create a
migrator.properties
file. Configure the source cache store in
migrator.properties
.Prepend all configuration properties with
source.
as in the following example:source.type=SOFT_INDEX_FILE_STORE source.cache_name=myCache source.location=/path/to/source/sifs
Configure the target cache store in
migrator.properties
.Prepend all configuration properties with
target.
as in the following example:target.type=SINGLE_FILE_STORE target.cache_name=myCache target.location=/path/to/target/sfs.dat
2.3.1. Store Migrator Properties
Configure source and target cache stores in a StoreMigrator
properties.
Table 2.1. Cache Store Type Property
Property | Description | Required/Optional |
---|---|---|
| Specifies the type of cache store type for a source or target.
| Required |
Table 2.2. Common Properties
Property | Description | Example Value | Required/Optional |
---|---|---|---|
| Names the cache that the store backs. |
| Required |
| Specifies the number of segments for target cache stores that can use segmentation.
The number of segments must match In other words, the number of segments for a cache store must match the number of segments for the corresponding cache. If the number of segments is not the same, Data Grid cannot read data from the cache store. |
| Optional |
Table 2.3. JDBC Properties
Property | Description | Required/Optional |
---|---|---|
| Specifies the dialect of the underlying database. | Required |
| Specifies the marshaller version for source cache stores. Set one of the following values:
*
*
* | Required for source stores only.
For example: |
| Specifies a custom marshaller class. | Required if using custom marshallers. |
|
Specifies a comma-separated list of custom | Optional |
| Specifies the JDBC connection URL. | Required |
| Specifies the class of the JDBC driver. | Required |
| Specifies a database username. | Required |
| Specifies a password for the database username. | Required |
| Sets the database major version. | Optional |
| Sets the database minor version. | Optional |
| Disables database upsert. | Optional |
| Specifies if table indexes are created. | Optional |
| Specifies additional prefixes for the table name. | Optional |
| Specifies the column name. | Required |
| Specifies the column type. | Required |
|
Specifies the | Optional |
To migrate from Binary cache stores in older Data Grid versions, change table.string.*
to table.binary.\*
in the following properties:
-
source.table.binary.table_name_prefix
-
source.table.binary.<id\|data\|timestamp>.name
-
source.table.binary.<id\|data\|timestamp>.type
# Example configuration for migrating to a JDBC String-Based cache store target.type=STRING target.cache_name=myCache target.dialect=POSTGRES target.marshaller.class=org.example.CustomMarshaller target.marshaller.externalizers=25:Externalizer1,org.example.Externalizer2 target.connection_pool.connection_url=jdbc:postgresql:postgres target.connection_pool.driver_class=org.postrgesql.Driver target.connection_pool.username=postgres target.connection_pool.password=redhat target.db.major_version=9 target.db.minor_version=5 target.db.disable_upsert=false target.db.disable_indexing=false target.table.string.table_name_prefix=tablePrefix target.table.string.id.name=id_column target.table.string.data.name=datum_column target.table.string.timestamp.name=timestamp_column target.table.string.id.type=VARCHAR target.table.string.data.type=bytea target.table.string.timestamp.type=BIGINT target.key_to_string_mapper=org.infinispan.persistence.keymappers. DefaultTwoWayKey2StringMapper
Table 2.4. RocksDB Properties
Property | Description | Required/Optional |
---|---|---|
| Sets the database directory. | Required |
| Specifies the compression type to use. | Optional |
# Example configuration for migrating from a RocksDB cache store. source.type=ROCKSDB source.cache_name=myCache source.location=/path/to/rocksdb/database source.compression=SNAPPY
Table 2.5. SingleFileStore Properties
Property | Description | Required/Optional |
---|---|---|
|
Sets the directory that contains the cache store | Required |
# Example configuration for migrating to a Single File cache store. target.type=SINGLE_FILE_STORE target.cache_name=myCache target.location=/path/to/sfs.dat
Table 2.6. SoftIndexFileStore Properties
Property | Description | Value |
---|---|---|
Required/Optional |
| Sets the database directory. |
Required |
| Sets the database index directory. |
# Example configuration for migrating to a Soft-Index File cache store. target.type=SOFT_INDEX_FILE_STORE target.cache_name=myCache target.location=path/to/sifs/database target.location=path/to/sifs/index
2.4. Migrating Cache Stores
Run StoreMigrator
to migrate data from one cache store to another.
Prerequisites
-
Get
infinispan-tools.jar
. -
Create a
migrator.properties
file that configures the source and target cache stores.
Procedure
If you build
infinispan-tools.jar
from source, do the following:-
Add
infinispan-tools.jar
and dependencies for your source and target databases, such as JDBC drivers, to your classpath. -
Specify
migrator.properties
file as an argument forStoreMigrator
.
-
Add
If you pull
infinispan-tools.jar
from the Maven repository, run the following command:mvn exec:java