How to change the number of pulp workers on Red Hat Satellite 6?

Solution Verified - Updated -

Environment

  • Red Hat Satellite 6
  • Capsule 6

Issue

  • There is a need to adjust number of worker processes in pulp
    • default value is number of processors on the machine
  • How to do that?

Resolution

Do NOT follow this tuning until specifically requested by Red Hat Support. Misconfiguring pulp might have negative impact to performance or stability of pulp server within Satellite or Capsule.

The below procedure works for Satellite as well as Capsule. To set e.g. 8 worker processes on each installer run:

  • For Satellite/Capsule 6.10 or newer(this will create the entry pulpcore_worker_count in /etc/foreman-installer/scenarios.d/satellite-answers.yaml):
# satellite-installer --foreman-proxy-content-pulpcore-worker-count=8

  • For Satellite 6.4-6.9:

    # satellite-installer -S satellite --katello-num-pulp-workers 8

  • For Capsule 6.4-6.9:

    # satellite-installer -S capsule --foreman-proxy-content-pulp-num-workers 8

Temporary change the number of workers, until next installer run reverts the change (for Satellite 6.4-6.9):

  • edit /etc/default/pulp_workers and place there:
PULP_CONCURRENCY=8
  • to apply the change, restart all pulp services:
# for i in pulp_resource_manager pulp_workers pulp_celerybeat; do service $i restart; done

IMPORTANT

  • In case one decreases pulp worker processes in Satellite 6.4-6.9, be aware of this pulp upstream bug. To workaround it, first stop all pulp services, ensure no process with celery in the command line is running (ps aux | grep python | grep -c celery returns 0), apply the change and then start the processes again.
  • Be careful of increasing pulp workers to more than 8 even if server having more than 8 CPU's since I/O could become a bottleneck, esp. when other I/O demanding features are concurrently used. If required to have more workers than 8, it is recommended to experiment with e.g. 10 workers, check possible impact, then increase to 12 and see if I/O becomes a problem etc.

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 2.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 2.0-related Issues

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 3.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 3.0-related Issues

Diagnostic Steps

  • With a system with less than 8 CPUs, PULP_CONCURRENCY is set to the number of CPUs. With a system with more than 8 CPUs, it is set to 8.
FROM /usr/share/katello-installer-base/pulp/spec/classes/pulp_config_spec.rb


    describe 'with processor count less than 8' do

      it "should set the PULP_CONCURRENCY to the processor count" do
        should contain_file('/etc/default/pulp_workers').with_content(/^PULP_CONCURRENCY=3$/)
      end

    end

    describe 'with processor count more than 8' do
      let :facts do
        default_facts.merge({
          :processorcount => 12
        })
      end

      it "should set the PULP_CONCURRENCY to 8" do
        should contain_file('/etc/default/pulp_workers').with_content(/^PULP_CONCURRENCY=8$/)
      end
    end

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments