Appendix A. Large Deployment Considerations

Increasing the Maximum Number of File Descriptors for Apache

With more than 800 content hosts registered, Apache can reach several system-level limits, resulting in new content host registration failure. To avoid this, file descriptor limits must be increased before deploying a large number of content hosts.

  1. Create the /etc/systemd/system/httpd.service.d/limits.conf file and insert the following text:

    [Service]
    LimitNOFILE=65536
  2. Apply the changes to the unit.

    # systemctl daemon-reload
  3. Restart Satellite services.

    # foreman-maintain service restart

Increasing the Maximum Number of File Descriptors for qpid

With more than 1100 content hosts with goferd running for errata updates, the qpid reach system-level limits, resulting in registration failures. To avoid this, file descriptors limits must be increased before deploying a large number of content hosts.

Increasing the Maximum Number of File Descriptors for qpid

  1. Create the /etc/systemd/system/qpidd.service.d/limits.conf file and insert the following text:

    [Service]
    LimitNOFILE=65536
  2. Apply the changes to the unit.

    # systemctl daemon-reload
    # systemctl restart qpidd.service

Increasing the Shared Buffer and Work Memory

You can increase the shared_buffer and work_mem to 256M and 4M respectively.

  1. On Red Hat Enterprise Linux 7, create the /var/lib/pgsql/data/postgresql.conf file and insert the following text:

    work_mem = 4MB
    shared_buffers = 256MB
  2. Restart postgresql services.

    # systemctl restart postgresql

Increasing Concurrent Content Host Registrations

To avoid reaching system-level limits, you can increase the global passenger queue limit to accommodate up to 250 concurrent content hosts.

To increase the global passenger queue limit, complete the following steps:

  1. Adjust the maximum passenger pool size to 1.5 times the physical CPU cores available to the Satellite Server.

    For example, if you have a Satellite Server with 16 cores, then the maximum passenger pool size is 24. This number is referenced as an example and you should use the number applicable to your environment.

  2. Edit the /etc/httpd/conf.d/passenger.conf file, updating the IfModule stanza to match the following text:

    <IfModule mod_passenger.c>
      PassengerRoot /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/locations.ini
      PassengerRuby /usr/bin/ruby
      PassengerMaxPoolSize 24
      PassengerMaxRequestQueueSize 200
      PassengerStatThrottleRate 120
    </IfModule>
  3. Edit the Foreman Passenger application configuration file /etc/httpd/conf.d/05-foreman-ssl.conf, updating the stanza starting with PassengerAppRoot to match the following text:

     PassengerAppRoot /usr/share/foreman
     PassengerRuby /usr/bin/tfm-ruby
     PassengerMinInstances 6
     PassengerStartTimeout 90
     PassengerMaxPreloaderIdleTime 0
     PassengerMaxRequests 10000
     PassengerPreStart https://example.com
  4. Edit the Puppet Passenger application configuration file /etc/httpd/conf.d/25-puppet.conf, adding the following text to the end of the virtual host definition:

     PassengerMinInstances 6
     PassengerStartTimeout 90
     PassengerMaxPreloaderIdleTime 0
     PassengerMaxRequests 10000
     PassengerPreStart https://example.com:8140
  5. Change the maximum connections in the /var/lib/pgsql/data/postgresql.conf file.

    max_connections = 500
  6. Restart postgresql services.

    # systemctl restart postgresql

Increasing the maximum number of open files for qdrouterd

With more than 1000 content hosts registered, qdrouterd can reach the default maximum number of open files. To avoid this, increase the maximum number of open files on the Satellite Server and all external Capsule Servers.

  1. Calculate the required maximum number of open files, using the following equation.

    (3 x number of content hosts) + 100

    For example, with 1020 content hosts, the new maximum should be set to 3160 ((3 x 1020) + 100).

  2. Create the file /etc/systemd/system/qdrouterd.service.d/limits.conf and add the following text.

    [Service]
    LimitNOFILE=maximum_number_of_files
    1. Apply the changes to the unit.

      # systemctl daemon-reload
    2. Restart Satellite services.

      # foreman-maintain service restart

Reducing delays in host registration

Communication between Satellite and each registered host is secured by use of certificates. When a host is registered, Satellite creates two certificates, an identity certificate and a Puppet certificate. The algorithm used to create each certificate requires random data from the Red Hat Enterprise Linux kernel. If not enough entropy is available when a host is registered, there is a delay until a suitable level of entropy is available. In very large environments, with more than 10,000 hosts, the rate of host registration is likely to be slowed by the lack of entropy. Several methods can be used to improve the availability of entropy to the Linux kernel, and so reduce the risk to performance of Satellite.

By default, the Linux kernel uses the /dev/random device as the source of random numbers. This is a blocking device, which means it stops supplying numbers when it determines that the amount of entropy is insufficient for generating a properly random output. It is this wait time which causes the delay in registering hosts. To resolve this issue, use the /dev/urandom device, as this is a non-blocking device.

Some hardware servers have processors which include hardware random number generators. For those that are supported by the Red Hat Enterprise Linux kernel, you can use that as the source of random numbers. For more information, see the hardware vendor’s documentation.

If Satellite is hosted on a virtual machine, note that some hypervisors make the hardware server’s random number generator available to the virtual machines it hosts. Red Hat Virtualization features the virtio RNG (Random Number Generator) device that provides KVM virtual machines access to entropy from the Red Hat Virtualization Host. On guests running Red Hat Enterprise Linux 7.0, you must install and configure rngd. On guests running Red Hat Enterprise Linux 7.1 and later, the guest kernel fetches entropy from the host as required. If a host’s random number generator is shared by guests, use of a hardware random number generator is recommended.

For more information about random number generators for guests, see Random Number Generator Device in the Red Hat Enterprise Linux 7 Virtualization Deployment and Administration Guide. For other hypervisors, see the vendor’s documentation.

For more information about the random number generator daemon, rngd, see Using the Random Number Generator in the Red Hat Enterprise Linux 7 Security Guide.