Chapter 8. Georeplication of storage in Red Hat Quay

Georeplication allows for a single globally-distributed Red Hat Quay to serve container images from localized storage.

When georeplication is configured, container image pushes will be written to the preferred storage engine for that Red Hat Quay instance. After the initial push, image data will be replicated in the background to other storage engines. The list of replication locations is configurable. An image pull will always use the closest available storage engine, to maximize pull performance.

8.1. Prerequisites

Georeplication requires that there be a high availability storage engine (S3, GCS, RADOS, Swift) in each geographic region. Further, each region must be able to access every storage engine due to replication requirements.

Note

Local disk storage is not compatible with georeplication at this time.

8.2. Visit the Management Panel

Sign in to a superuser account from the Red Hat Quay login screen. For example, if the host were reg.example.com, you would go to http://reg.example.com/superuser to view the management panel: Log in as superuser to set up Georeplication

8.3. Enable storage replication

  1. Click the configuration tab and scroll down to the section entitled Registry Storage.
  2. Click Enable Storage Replication.
  3. Add each of the storage engines to which data will be replicated. All storage engines to be used must be listed.
  4. If complete replication of all images to all storage engines is required, under each storage engine configuration click Replicate to storage engine by default. This will ensure that all images are replicated to that storage engine. To instead enable per-namespace replication, please contact support.
  5. Click Save to validate.

  6. After adding storage and enabling “Replicate to storage engine by default” for Georeplications, you need to sync existing image data across all storage. To do this, you need to oc exec (or docker/kubectl exec) into the container and run: 

    $ venv/bin/python -m util.backfillreplication

    This is a one time operation to sync content after adding new storage.

8.4. Run Red Hat Quay with storage preferences

  1. Copy the config.yaml to all machines running Red Hat Quay

  2. For each machine in each region, add a QUAY_DISTRIBUTED_STORAGE_PREFERENCE environment variable with the preferred storage engine for the region in which the machine is running.

    For example, for a machine running in Europe with the config directory on the host available from /mnt/quay/config: 

    # docker run -d -p 443:443 -p 80:80 -v /mnt/quay/config:/conf/stack \
    -e QUAY_DISTRIBUTED_STORAGE_PREFERENCE=europestorage \
    quay.io/coreos/quay:versiontag
    Note

    The value of the environment variable specified must match the name of a Location ID as defined in the config panel.

  3. Restart all Quay containers