Chapter 12. Georeplication of storage in Red Hat Quay

Georeplication allows for a single globally-distributed Red Hat Quay to serve container images from localized storage.

When georeplication is configured, container image pushes will be written to the preferred storage engine for that Red Hat Quay instance. After the initial push, image data will be replicated in the background to other storage engines. The list of replication locations is configurable. An image pull will always use the closest available storage engine, to maximize pull performance.

12.1. Prerequisites

Georeplication requires that there be a high availability storage engine (S3, GCS, RADOS, Swift) in each geographic region. Further, each region must be able to access every storage engine due to replication requirements.

Note

Local disk storage is not compatible with georeplication at this time.

12.2. Visit the Config Tool

Open the Red Hat Quay Config Tool to configure storage for georeplication.

12.3. Enable storage replication

  1. Scroll down to the section entitled Registry Storage.
  2. Click Enable Storage Replication.
  3. Add each of the storage engines to which data will be replicated. All storage engines to be used must be listed.
  4. If complete replication of all images to all storage engines is required, under each storage engine configuration click Replicate to storage engine by default. This will ensure that all images are replicated to that storage engine. To instead enable per-namespace replication, please contact support.
  5. When you are done, click Save Configuration Changes. Configuration changes will take effect the next time Red Hat Quay restarts.
  6. After adding storage and enabling “Replicate to storage engine by default” for Georeplications, you need to sync existing image data across all storage. To do this, you need to oc exec (or docker/kubectl exec) into the container and run:

    # scl enable python27 bash
    # python -m util.backfillreplication

    This is a one time operation to sync content after adding new storage.

12.4. Run Red Hat Quay with storage preferences

  1. Copy the config.yaml to all machines running Red Hat Quay
  2. For each machine in each region, add a QUAY_DISTRIBUTED_STORAGE_PREFERENCE environment variable with the preferred storage engine for the region in which the machine is running.

    For example, for a machine running in Europe with the config directory on the host available from /mnt/quay/config:

    # sudo podman login quay.io
    Username: yourquayuser
    Password: *****
    # sudo podman run -d -p 443:8443 -p 8080:8080 -v /mnt/quay/config:/conf/stack:Z \
        -e QUAY_DISTRIBUTED_STORAGE_PREFERENCE=europestorage \
        registry.redhat.io/quay/quay-rhel8:v3.5.0
    Note

    The value of the environment variable specified must match the name of a Location ID as defined in the config panel.

  3. Restart all Red Hat Quay containers