Satellite 5.7 performance issues after upgrade
Has anyone notice a drop in performance after upgrading to the Satellete 5.7 from 5.6? i get a big lag in time, from scheduling a task
and the time it takes to show up in under scheduling as a pending action. Also, I have noticed a client shows up in pending actions, but
when I go on the server and to a rhn_check -v. The command completes with nothing to report, yet on the web UI still shows up in pending
actions. It seems to me it might be a database performance issue, as it was an upgrade from PostgresSQL 8.4 to 9.2.
For the community are there any peformance tools for Satellite 5.7 or PostgresSQL 9.2 tuning settings for Satellite?
Responses
I upgraded to 5.7 on Wednesday and now I get a lot of 503 errors. On thing in particular that causes 503 errors is trying to run a remote job on "a bunch" of servers at once. I've successfully scheduled a remote job on 500 servers, but if I try 1000 I get a long delay ending in a 503 error when I click on "Run remote commands" from the System Set Manager page. I never even get the chance to specify the commands to run, or schedule them.
Kind of makes 5.7 useless to me if I can't use it to manage a lot of servers at once. :(
I've got one 8 CPU, 16 GB RAM RHEL 6 VM for the postgres 8.4 database server that has one postmaster process running at 100% (from top) but the system as a whole is ~85% idle.
My Satellite server is also a 8 CPU, 16 GB RAM RHEL 6 VM with one tomcat owned java process running at over 100% (again, from top), but the server as a whole is also ~85% idle.
Is there a good way to get the system to take advantage of the other processors, or is the nature of Satellite too single threaded for that?
There is a delay now in some of the details logged for each client. I thought it was mainly the package list though. When you run a bunch of updates, previous Satellite's would show the system as being up to date immediately on completion of the task(s). Now, however, the update of the package profile is done by a batch task in the background (forget which one). The frequency of this can be increased, but there's still a lag.
Wouldn't expect the Pending/Completed tasks to be affected in this way though. Was disappointed enough with the package profiles.
numerous clients coming to pick up jobs will come through osa-dispatcher if you've configured push-to-client. I'm not sure how best to optimise that. But I've increased the number of apache processes in the past to assist with redeploying lots of machines simultaneously. That allowed me to increase overall CPU usage on the Satellite and max out the network bandwidth properly.
I've noticed with older Satellites that a scheduling tasks on large numbers of machines appears to grind the Satellite initially, but this always sorts itself out in 30 seconds or so. I'm migrating to Satellite 5.7 at the moment, but haven't done the live migration, so don't have more than a handful of clients registered to the 5.7 Satellite at any one point.
I know this is a few months old.
Some of the info never seems to get updated properly.
Package info is updated fine but the hardware info isn't. There is a tool called rhn-profile-sync that performs package and hardware updates. I've added a crontab to each system that talks to the satellite that performs the sync once a month. I've set the crontab to run at a different time on each system.
Here's an example of how I set it up:
echo echo $(( RANDOM % (59 - 0 + 1 ) + 0 )) "00 01 * * /usr/sbin/rhn-profile-sync" >> /var/spool/cron/root
We have 4000+ systems on our Satellite, so this way there aren't thousands of requests all coming in within a couple of minutes. It's spread out over an hour.
As far as tuning Tomcat goes, increase the Xms and Xmx in /etc/sysconfig/tpmcat6. I've set them to 1024m and 2048m respectively.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
