Chapter 8. Redis high availability (HA) support for 3scale

Note

There are known issues with Redis high availability (HA) support for 3scale. For more information, see the Red Hat 3scale API Management 2.5 release notes, Chapter 8. Known issues in the release notes.

8.1. Introduction

High availability (HA) is provided for most components by the OpenShift Container Platform (OCP). For more information see OpenShift Container Platform 3.11 Chapter 30. High Availability.

The database components for HA in 3scale include:

  • system-redis: provides temporary storage for background jobs for 3scale and it is also used as a message bus for Ruby processes of system-app pods.
  • backend-redis: used for statistics storage and temporary job storage.
Note

Both system-redis and backend-redis can be replaced by the Redis Cluster (open-source or Redis Labs).

The following env vars can be set into system-(app,sidekiq,sphinx) deployment configurations, though it is only a requirement for Redis Enterprise:

  • MESSAGE_BUS_REDIS_URL (a redis URL)
  • REDIS_NAMESPACE (a short string to namespace Sidekiq’s Redis keys)
  • MESSAGE_BUS_REDIS_NAMESPACE (a short string to namespace System message bus’s Redis keys)

A new pod is created automatically when the Redis pod dies or is killed by OCP and the data is restored from the persistent storage, so the pod continues to work. In the scenario described, there would be a small amount of downtime while the new pod is being started. This is due to the limitation that Redis does not support a multi-master setup. Downtime can be reduced by preloading the Redis images onto all nodes that have Redis deployed to them, which will speed up the pod restart.

8.2. Setting up Redis for zero downtime

If zero downtime is required, Redis would need to be set up outside of OCP. There are several ways to set it up using the configuration options of 3scale pods:

Note

Red Hat does not provide support for the above mentioned services. The mention of any such services does not imply endorsement by Red Hat of the products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) any external content.

8.3. Configurating backend components for 3scale

There are configuration settings in 3scale 2.5 to configure Redis HA (failover) for the backend component. They can be set as environment variables in the following deployment configurations: backend-cron, backend-listener, and backend-worker:

  • CONFIG_REDIS_SENTINEL_HOSTS and CONFIG_QUEUES_SENTINEL_HOSTS:

    A comma-separated list of Sentinel hosts for the main statistics database and the Resque background job database.

    Note

    Values should be in the format: name:value <host>:<port> For example: host1:26379, host2:26379, or host3:26379

  • CONFIG_REDIS_SENTINEL_ROLE and CONFIG_QUEUES_SENTINEL_ROLE:

    The role of each Sentinels group, either master or slave. Currently only master (default) is supported.

This makes the value of CONFIG_REDIS_PROXY and CONFIG_QUEUES_MASTER_NAME take the meaning of Sentinel group name instead of a specific server.

  • When no Sentinel hosts are configured, the environment variables CONFIG_REDIS_PROXY and CONFIG_QUEUES_MASTER_NAME can use URLs and support password-protected databases, for example: CONFIG_REDIS_PROXY=redis://user:password@server:port/database

    • The connection is then established with the password-protected Redis instance.
  • When Sentinel hosts are configured, the password must be set in the Sentinel configuration and use the Sentinel group name instead: CONFIG_REDIS_PROXY=master_group