Chapter 1. Advanced Red Hat Quay configuration
You can configure your Red Hat Quay after initial deployment using several different interfaces:
The Red Hat Quay Config Tool: Running the
configmode presents a Web-based interface for configuring the Red Hat Quay cluster. This is the recommended method for most configuration of the Red Hat Quay service itself.
config.yamlfile holds most of the configuration information for the Red Hat Quay cluster. Editing that file directly is possible, but it is only recommended for advanced tuning and performance features that are not available through the Config Tool.
- Red Hat Quay API: Some Red Hat Quay configuration can be done through the API.
While configuration for specific features is covered in separate sections, this section describes how to use each of those interfaces and perform some more advanced configuration.
1.1. Using Red Hat Quay Config Tool to modify Red Hat Quay
The Red Hat Quay Config Tool is made available by running a
Quay container in
config mode alongside the regular Red Hat Quay service. Running the Config Tool is different for Red Hat Quay clusters running on OpenShift than it is for those running directly on host systems.
1.1.1. Running the Config Tool from the Red Hat Quay Operator
If you are running the Red Hat Quay Operator from OpenShift, the Config Tool is probably already available for you to use. To access the Config Tool, do the following:
- From the OpenShift console, select the project in which Red Hat Quay is running. For example, quay-enterprise.
From the left column, select Networking → Routes. You should see routes to both the Red Hat Quay application and Config Tool, as shown in the following image:
- Select the route to the Config Tool (for example, example-quayecosystem-quay-config) and select it. The Config tool Web UI should open in your browser.
Modify configuration for this cluster. You should see the Config Tool, ready for you to change features of your Red Hat Quay cluster, as shown in the following image:
When you have made the changes you want, select
Save Configuration Changes. The Config Tool will validate your changes.
Make any corrections as needed by selecting
Continue Editingor select
Nextto continue on.
When prompted, it is recommended that you select
Download Configuration. That will download a tarball of your new
config.yaml, as well as any certificates and keys used with your Red Hat Quay setup.
Go to deployment rollout, then
Populate the configuration to deployments. The Red Hat Quay pods will be restarted and the changes will take effect.
config.yaml file you saved can be used to make advanced changes to your configuration or just kept for future reference.
1.1.2. Running the Config Tool from the command line
If you are running Red Hat Quay directly from a host system, using tools such as the
docker commands, after the initial Red Hat Quay deployment, you can restart the Config Tool to modify your Red Hat Quay cluster. Here’s how:
Start quay in config mode: On the first
quaynode run the following, replacing
my-secret-passwordwith your password. If you would like to modify an existing config bundle, you can simply mount your configuration directory into the
Quaycontainer as you would in registry mode.
# podman run --rm -it --name quay_config -p 8080:8080 \ -v path/to/config-bundle:/conf/stack \ registry.redhat.io/quay/quay-rhel8:v3.5.6 config my-secret-password
- Open browser: When the quay configuration tool starts up, open a browser to the URL and port 8080 of the system you are running the configuration tool on (for example https://myquay.example.com:8080). You are prompted for a username and password.
At this point, you can begin modifying your Red Hat Quay cluster as described earlier.
1.2. Using the API to modify Red Hat Quay
See the Red Hat Quay API Guide for information on how to access Red Hat Quay API.
1.3. Editing the
config.yaml file to modify Red Hat Quay
Some advanced Red Hat Quay configuration that is not available through the Config Tool can be achieved by editing the
config.yaml file directly. Available settings are described in the Schema for Red Hat Quay configuration The following are examples of settings you can change directly in the
1.3.1. Add name and company to Red Hat Quay sign-in
Setting the following will cause users to be prompted for their name and company when they first sign in. Although this is optional, it can provide you with extra data about your Red Hat Quay users:
1.3.2. Disable TLS Protocols
You can change the SSL_PROTOCOLS setting to remove SSL protocols that you do not want to support in your Red Hat Quay instance. For example, to remove TLS v1 support from the default SSL_PROTOCOLS : ['TLSv1','TLSv1.1','TLSv1.2'], change it as follows:
+ SSL_PROTOCOLS : ['TLSv1.1','TLSv1.2']
1.3.3. Rate limit API calls
Adding the FEATURE_RATE_LIMITS parameter to the
nginx to limit certain API calls to 30 per second. If that feature is not set, API calls are limied to 300 per second (effectively unlimited). Rate limiting can be an important feature, if you need to make sure the resources available are not overwhelmed with traffic.
Some namespace may require unlimited access (perhaps they are important to CI/CD and take priority, for example). In this case, those namespace may be placed in a list in
config.yaml for NON_RATE_LIMITED_NAMESPACES.
1.3.4. Adjust database connection pooling
Red Hat Quay is composed of many different processes which all run within the same container. Many of these processes interact with the database.
If enabled, each process that interacts with the database will contain a connection pool. These per-process connection pools are configured to maintain a maximum of 20 connections. Under heavy load, it is possible to fill the connection pool for every process within a Red Hat Quay container. Under certain deployments and loads, this may require analysis to ensure Red Hat Quay does not exceed the database’s configured maximum connection count.
Overtime, the connection pools will release idle connections. To release all connections immediately, Red Hat Quay requires a restart.
Database connection pooling may be toggled by setting the environment variable
If database connection pooling is enabled, it is possible to change the maximum size of the connection pool. This can be done through the following
DB_CONNECTION_ARGS: max_connections: 10
188.8.131.52. Database connection arguments
You can customize Red Hat Quay database connection settings within the
config.yaml file. These are entirely dependent upon the underlying database driver, such as
psycopg2 for Postgres and
pymysql for MySQL. It is also possible to pass in arguments used by Peewee’s Connection Pooling mechanism as seen below.
DB_CONNECTION_ARGS: max_connections: n # Max Connection Pool size. (Connection Pooling only) timeout: n # Time to hold on to connections. (Connection Pooling only) stale_timeout: n # Number of seconds to block when the pool is full. (Connection Pooling only)
184.108.40.206. Database SSL configuration
Some key-value pairs defined under DB_CONNECTION_ARGS are generic while others are database-specific. In particular, SSL configuration depends on the database you are deploying.
220.127.116.11.1. PostgreSQL SSL connection arguments
A sample PostgreSQL SSL configuration is given below:
DB_CONNECTION_ARGS: sslmode: verify-ca sslrootcert: /path/to/cacert
sslmode option determines whether or with what priority a secure SSL TCP/IP connection will be negotiated with the server. There are six modes:
- disable: only try a non-SSL connection
- allow: first try a non-SSL connection; if that fails, try an SSL connection
- prefer: (default) first try an SSL connection; if that fails, try a non-SSL connection
- require: only try an SSL connection. If a root CA file is present, verify the certificate in the same way as if verify-ca was specified
- verify-ca: only try an SSL connection, and verify that the server certificate is issued by a trusted certificate authority (CA)
- verify-full: only try an SSL connection, verify that the server certificate is issued by a trusted CA and that the requested server host name matches that in the certificate
More information on the valid arguments for PostgreSQL is available at https://www.postgresql.org/docs/current/libpq-connect.html.
18.104.22.168.2. MySQL SSL connection arguments
A sample MySQL SSL configuration follows:
DB_CONNECTION_ARGS: ssl: ca: /path/to/cacert
Information on the valid connection arguments for MySQL is available at https://dev.mysql.com/doc/refman/8.0/en/connecting-using-uri-or-key-value-pairs.html.
22.214.171.124. HTTP connection counts
It is possible to specify the quantity of simultaneous HTTP connections using environment variables. These can be specified as a whole, or for a specific component. The default for each is 50 parallel connections per process.
WORKER_CONNECTION_COUNT_REGISTRY=n WORKER_CONNECTION_COUNT_WEB=n WORKER_CONNECTION_COUNT_SECSCAN=n WORKER_CONNECTION_COUNT=n
Specifying a count for a specific component will override any value set in WORKER_CONNECTION_COUNT.
126.96.36.199. Dynamic process counts
To estimate the quantity of dynamically sized processes, the following calculation is used by default.
Red Hat Quay queries the available CPU count from the entire machine. Any limits applied using kubernetes or other non-virtualized mechanisms will not affect this behavior; Red Hat Quay will makes its calculation based on the total number of processors on the Node. The default values listed are simply targets, but shall not exceed the maximum or be lower than the minimum.
Each of the following process quantities can be overridden using the environment variable specified below.
registry - Provides HTTP endpoints to handle registry action
- minimum: 8
- maximum: 64
- default: $CPU_COUNT x 4
- environment variable: WORKER_COUNT_REGISTRY
web - Provides HTTP endpoints for the web-based interface
- minimum: 2
- maximum: 32
- default: $CPU_COUNT x 2
- environment_variable: WORKER_COUNT_WEB
secscan - Interacts with Clair
- minimum: 2
- maximum: 4
- default: $CPU_COUNT x 2
- environment variable: WORKER_COUNT_SECSCAN
188.8.131.52. Environment variables
Red Hat Quay allows overriding default behavior using environment variables. This table lists and describes each variable and the values they can expect.
Table 1.1. Worker count environment variables
Specifies the number of processes to handle Registry requests within the
Integer between 8 and 64
Specifies the number of processes to handle UI/Web requests within the container.
Integer between 2 and 32
Specifies the number of processes to handle Security Scanning (e.g. Clair) integration within the container.
Integer between 2 and 4
Toggle database connection pooling. In 3.4, it is disabled by default.
"true" or "false"
184.108.40.206. Turning off connection pooling
Red Hat Quay deployments with a large amount of user activity can regularly hit the 2k maximum database connection limit. In these cases, connection pooling, which is enabled by default for Red Hat Quay, can cause database connection count to rise exponentially and require you to turn off connection pooling.
If turning off connection pooling is not enough to prevent hitting that 2k database connection limit, you need to take additional steps to deal with the problem. In this case you might need to increase the maximum database connections to better suit your workload.