Satellite 5.4.1 unresponsive after cloning a channel

Latest response

I am using the satellite API to clone a channel. The Channel clones fine, but satellite becomes unresponsive and throws 500 errors. What I have found is restarting tomcat 6 resolves the problem, so it looks like it a problem with the ajp connection between apache and tomcat. The only error I can find is in the apache log file:

[Tue Mar 11 11:48:50 2014] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header
[Tue Mar 11 12:44:19 2014] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header
[Tue Mar 11 12:45:03 2014] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header

Has anyone else experienced this? Any ideas on how to correct the problem?

JR
Log in to join the conversation

Responses

james.radtke@siriusxm.com's picture Guru 6863 points
11 March 2014 6:37 PM

This is COMPLETE speculation... I wonder if you have exhausted the connection pool for the Web Service, or the JVM heap size?

Either way: I recommend opening a ticket and a backup ;-)

I would also check the logs in
/var/log/rhn

Increase the taskomatic JVM maximum heap size from 512Mb to 1.3Gb to stop the heap dumps from occurring.
Find the following values in /etc/rhn/default/rhn_taskomatic_daemon.conf:

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=256

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=512

Change the value of "wrapper.java.maxmemory" to 1.3Gb, like so:
# Initial Java Heap Size (in MB)
wrapper.java.initmemory=256

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=1331

There is a tunable for the Web Server, but I do not recall where that file was updated.

R. Hinton's picture Guru 6827 points
11 March 2014 11:47 PM

Joseph, see James' good tips,

Given the bit of searching I did on your errors and the potential "fixes" -- I'd agree with James and highly recommend you open a case with Red Hat support.

Do you have the freedom to go to Satellite version 5.6?

I'm currently taking the last of the 5.4.1 satellite servers we had and putting it (reload) to version 5.6 this week to join the others at 5.6.

ir. Jan Gerrit Kootstra's picture Guru 6222 points
11 March 2014 10:22 PM

I have this too since the last week.
Also messages about "Non empty fields being empty".
In my case for RH Satellite 5.6

JR Community Member 75 points
12 March 2014 1:24 PM

Thanks for the advice. I increased the memory as suggested and initially found the same problem. So I began tailing the log files in /var/log/rhn and found that even though the channels were created, it looks like there was still some work being done. I saw packages being assigned to the channels for about 5 minutes after the channel was visible in the interface.

After allowing all the channel data to be updated, I was able to delete the the channels without freezing up the interface.

james.radtke@siriusxm.com's picture Guru 6863 points
12 March 2014 2:13 PM

Thanks for the update.
That is 1 thing I actually wanted to submit an RFI for:
- easier task manager/progress meter for "the system"

There are SO many things going on, it would be cool to see what was currently running. Also, there are a few jobs that you submit via API/WebUI that simply go off and do their thing, and you cannot really tell.

I have been cloning my channels from the CLI using spacewalk-clone-by-date and that provides a pretty good status.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.