Red Hat Training

A Red Hat training course is available for Red Hat Gluster Storage

6.5. Creating Distributed Replicated Volumes

Important

Creating distributed-replicated volume with replica count greater than 3 is under technology preview. Technology Preview features are not fully supported under Red Hat subscription level agreements (SLAs), may not be functionally complete, and are not intended for production use. However, these features provide early access to upcoming product innovations, enabling customers to test functionality and provide feedback during the development process. As Red Hat considers making future iterations of Technology Preview features generally available, we will provide commercially reasonable efforts to resolve any reported issues that customers experience when using these features.
Use distributed replicated volumes in environments where the requirement to scale storage, and high-reliability is critical. Distributed replicated volumes also offer improved read performance in most environments.

Note

The number of bricks must be a multiple of the replica count for a distributed replicated volume. Also, the order in which bricks are specified has a great effect on data protection. Each replica_count consecutive bricks in the list you give will form a replica set, with all replica sets combined into a distribute set. To ensure that replica-set members are not placed on the same node, list the first brick on every server, then the second brick on every server in the same order, and so on.
Prerequisites

A trusted storage pool has been created, as described in Section 5.1, “Adding Servers to the Trusted Storage Pool”.Understand how to start and stop volumes, as described in Section 6.10, “Starting Volumes”.

6.5.1. Creating Two-way Distributed Replicated Volumes

Two-way distributed replicated volume distributes and creates two copies of files across multiple bricks in the volume. The number of bricks must be equal to the replica count for a replicated volume. To protect against server and disk failures, it is recommended that the bricks of the volume are from different servers.
Illustration of a Two-way Distributed Replicated Volume

Figure 6.4. Illustration of a Two-way Distributed Replicated Volume

Creating two-way distributed replicated volumes
  1. Run the gluster volume create command to create the distributed replicated volume.
    The syntax is # gluster volume create NEW-VOLNAME [replica COUNT] [transport tcp | rdma | tcp,rdma] NEW-BRICK...
    The default value for transport is tcp. Other options can be passed such as auth.allow or auth.reject. See Section 8.1, “Configuring Volume Options” for a full list of parameters.

    Example 6.5. Four Node Distributed Replicated Volume with a Two-way Replication

    The order in which bricks are specified determines how bricks are replicated with each other. For example, first 2 bricks, where 2 is the replica count. In this scenario, the first two bricks specified replicate each other. If more bricks were specified, the next two bricks in sequence would replicate each other. 
    # gluster volume create test-volume replica 2 transport tcp server1:/exp1/brick server2:/exp2/brick server3:/exp3/brick server4:/exp4/brick
Creation of test-volume has been successful
Please start the volume to access data.

    Example 6.6. Six Node Distributed Replicated Volume with a Two-way Replication

    # gluster volume create test-volume replica 2 transport tcp server1:/exp1/brick server2:/exp2/brick server3:/exp3/brick server4:/exp4/brick server5:/exp5/brick server6:/exp6/brick
Creation of test-volume has been successful
Please start the volume to access data.
  2. Run # gluster volume start VOLNAME to start the volume. 
    # gluster volume start test-volume
Starting test-volume has been successful
  3. Run gluster volume info command to optionally display the volume information.

Important

You must set quorum on replicated volumes to prevent split-brain scenarios. For more information on setting client-side quorum, see Section 8.10.1.2, “Configuring Client-Side Quorum”

6.5.2. Creating Three-way Distributed Replicated Volumes

Three-way distributed replicated volume distributes and creates three copies of files across multiple bricks in the volume. The number of bricks must be equal to the replica count for a replicated volume. To protect against server and disk failures, it is recommended that the bricks of the volume are from different servers.
Synchronous three-way replication is now fully supported in Red Hat Storage. Three-way replication volumes are supported only on JBOD configuration.
Illustration of a Three-way Distributed Replicated Volume

Figure 6.5. Illustration of a Three-way Distributed Replicated Volume

Recommended configuration of three-way replication for geo-replicated volume

The recommended configuration for three-way replication is to have a minimum of three nodes or a multiples of three, as only a single brick out of the replica set is involved in syncing the files to the slave. It is expected that, all the bricks of a replica set are in different nodes. For each replica set, select the nodes for the bricks as per the first replica set. It is recommended not to have a brick along with its replica set from the same volume residing in the same node.

The following is an example of the recommended configuration: 
# gluster volume info master
Volume Name: master
Type: Distributed-Replicate
Volume ID: 6e2e447d-550d-44e4-85be-33b35933de3c
Status: Started
Snap Volume: no
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: fedora1:/bricks/brick0/b0 
Brick2: fedora2:/bricks/brick0/b0 
Brick3: fedora3:/bricks/brick0/b0 
Brick4: fedora1:/bricks/brick1/b1 
Brick5: fedora2:/bricks/brick1/b1 
Brick6: fedora3:/bricks/brick1/b1
Creating three-way distributed replicated volumes
  1. Run the gluster volume create command to create the distributed replicated volume.
    The syntax is # gluster volume create NEW-VOLNAME [replica COUNT] [transport tcp | rdma | tcp,rdma] NEW-BRICK...
    The default value for transport is tcp. Other options can be passed such as auth.allow or auth.reject. See Section 8.1, “Configuring Volume Options” for a full list of parameters.

    Example 6.7. Six Node Distributed Replicated Volume with a Three-way Replication

    The order in which bricks are specified determines how bricks are replicated with each other. For example, first 3 bricks, where 3 is the replica count. In this scenario, the first three bricks specified replicate each other. If more bricks were specified, the next three bricks in sequence would replicate each other. 
    # gluster volume create test-volume replica 3 transport tcp server1:/exp1/brick server2:/exp2/brick server3:/exp3/brick server4:/exp4/brick server5:/exp5/brick server6:/exp6/brick
Creation of test-volume has been successful
Please start the volume to access data.
  2. Run # gluster volume start VOLNAME to start the volume. 
    # gluster volume start test-volume
Starting test-volume has been successful
  3. Run gluster volume info command to optionally display the volume information.

Important

You must set client-side quorum on replicated volumes to prevent split-brain scenarios. For more information on setting client-side quorum, see Section 8.10.1.2, “Configuring Client-Side Quorum”