RH Satellite Sync took two months to sync updated packages with clients?

Latest response

So I've been downloading incremental ISOs from the RH website and importing them into my disconnected RH 5.6 Satellite and from there update my RHEL 5/6 servers. Doing this monthly.

At first the process was working great, however since early August, when loading the updated ISOs into the RH Satellite, none of the new packages were appearing to the clients. I kind of thought something wasn't right, however I've downloaded more ISOs and was going to try again before saying anything. However this morning, I came into work and logged into the RH Satellite and noticed that the clients are all finally seeing the errata and update packages. I have no idea why this is happening.

The Satellite is a VM living in VMWare with the following:
- 8 CPUs
- 16 GB of RAM
- Plenty of HD Space
- As of late July 2015, updates with the latest packages/errata.

The database is external, on another RHEL server which is a VM also living in VMWare and running Oracle.

Right now, there aren't any obvious issues on RH Satellite of the Oracle database as to why this is happening. It takes normal time, around 20 to 30 minutes to process and I write the output to script and don't get any errors.

However if I've been importing incremental ISOs monthly for August and September 2015 and they are finally showing up now, something isn't correct.

I found an older thread about the same type of situation: https://access.redhat.com/discussions/960443

However I wanted to see what other though.

thanks

Responses

I do not think discussion 960443 is relevant.
The problem is the 'errata cache' computation took to long. This is currently
processed by taskomatic. So, either taskomatic wasn't running and not processing
the requests, or there are simply too many requests to regenerate 'errata cache' in
the taskomatic queue.
Lately we saw customers with too many requests in the queue, however we are not sure,
how customers came to that state. We introduced a patch not to create duplicate entries
in the queue (for Sat5.7 only) to keep the queue shorter.
https://bugzilla.redhat.com/show_bug.cgi?id=1227335

There's a hotfix reuqest for Sat 5.6 as well in https://bugzilla.redhat.com/show_bug.cgi?id=1243948. Please, make sure with engineering the hotfix is still applicable on current Sat5.6 if you plan to apply it. In the future there's a risk it may not be applicable, or it may introduce other kind of issues.

Checking the RH Satellite, Taskomatic daemon is running:

[root@foo ~]# ps aux | grep -i taskomatic
root      4768  0.0  0.0  61192   804 pts/1    S+   10:48   0:00 grep -i taskomatic
root      6488  0.0  0.0  18876   612 ?        Sl   Oct01   0:00 /usr/bin/taskomaticd /usr/share/rhn/config-defaults/rhn_taskomatic_daemon.conf wrapper.pidfile=/var/run//taskomatic.pid wrapper.daemonize=TRUE
root      6519  0.4  6.4 1768968 1062532 ?     Sl   Oct01  33:14 /usr/bin/java -Dibm.dst.compatibility=true -Dcom.ibm.tools.attach.enable=no -Xms256m -Xmx1024m -Djava.library.path=/usr/lib:/usr/lib64:/usr/lib/oracle/10.2.0.4/client64/lib:/usr/lib/oracle/10.2.0.4/client/lib -classpath /usr/share/java/tanukiwrapper.jar:/usr/share/rhn/classes:/usr/share/java/struts.jar:/usr/share/java/jfreechart.jar:/usr/share/java/jpam.jar:/usr/share/java/javamail.jar:/usr/share/java/axis/axis-ant.jar:/usr/share/java/quartz.jar:/usr/share/java/commons-codec.jar:/usr/share/java/commons-beanutils.jar:/usr/share/java/ojdbc14.jar:/usr/share/java/jta.jar:/usr/share/java/concurrent.jar:/usr/share/rhn/lib/spacewalk-asm.jar:/usr/share/java/axis/jaxrpc.jar:/usr/share/java/commons-collections.jar:/usr/share/java/taglibs-standard.jar:/usr/share/java/axis/axis.jar:/usr/share/java/xalan-j2.jar:/usr/share/java/commons-validator.jar:/usr/share/java/asm/asm-attrs.jar:/usr/share/java/jaf.jar:/usr/share/java/jdom.jar:/usr/share/java/oro.jar:/usr/share/rhn/lib/rhn.jar:/usr/share/java/redstone-xmlrpc.jar:/usr/share/java/oscache.jar:/usr/share/java/log4j.jar:/usr/share/java/wsdl4j.jar:/usr/share/java/jcommon.jar:/usr/share/java/commons-el.jar:/usr/share/java/taglibs-core.jar:/usr/share/java/commons-lang.jar:/usr/share/java/commons-digester.jar:/usr/share/java/jasper5-runtime.jar:/usr/share/java/jspapi.jar:/usr/share/java/c3p0.jar:/usr/share/java/sitemesh.jar:/usr/share/java/jasper5-compiler.jar:/usr/share/java/axis/saaj.jar:/usr/share/java/commons-logging.jar:/usr/share/java/commons-discovery.jar:/usr/share/java/xml-commons-apis.jar:/usr/share/java/axis/jaxrpc.jar:/usr/share/java/commons-cli.jar:/usr/share/java/bcel.jar:/usr/share/java/antlr.jar:/usr/share/java/xerces-j2.jar:/usr/share/java/hibernate3-testsuite-3.3.2-sources.jar:/usr/share/java/hibernate3-core-3.3.2-sources.jar:/usr/share/java/hibernate3-oscache.jar:/usr/share/java/hibernate3-testing.jar:/usr/share/java/hibernate3-testsuite-3.3.2.jar:/usr/share/java/hibernate3-proxool-3.3.2-sources.jar:/usr/share/java/hibernate3-jbosscache2-3.3.2-sources.jar:/usr/share/java/hibernate3-jmx.jar:/usr/share/java/hibernate3-jdbc4-testing.jar:/usr/share/java/hibernate3-ehcache-3.3.2-sources.jar:/usr/share/java/hibernate3-core-3.3.2.jar:/usr/share/java/hibernate3-c3p0-3.3.2.jar:/usr/share/java/hibernate3-proxool.jar:/usr/share/java/hibernate3-jdbc4-testing-3.3.2-sources.jar:/usr/share/java/hibernate3-testsuite.jar:/usr/share/java/hibernate3-jbosscache2-3.3.2.jar:/usr/share/java/hibernate3-jmx-3.3.2-sources.jar:/usr/share/java/hibernate3-swarmcache-3.3.2.jar:/usr/share/java/hibernate3-jmx-3.3.2.jar:/usr/share/java/hibernate3-swarmcache-3.3.2-sources.jar:/usr/share/java/hibernate3-ehcache.jar:/usr/share/java/hibernate3-testing-3.3.2.jar:/usr/share/java/hibernate3-oscache-3.3.2.jar:/usr/share/java/hibernate3-proxool-3.3.2.jar:/usr/share/java/hibernate3-c3p0.jar:/usr/share/java/hibernate3-swarmcache.jar:/usr/share/java/hibernate3-ehcache-3.3.2.jar:/usr/share/java/hibernate3-c3p0-3.3.2-sources.jar:/usr/share/java/hibernate3-oscache-3.3.2-sources.jar:/usr/share/java/hibernate3-core.jar:/usr/share/java/hibernate3-jdbc4-testing-3.3.2.jar:/usr/share/java/hibernate3-jbosscache2.jar:/usr/share/java/hibernate3-testing-3.3.2-sources.jar:/usr/share/java/cglib.jar:/usr/share/java/dom4j.jar:/usr/share/rhn/lib/java-branding.jar:/usr/share/java/slf4j/api.jar:/usr/share/java/slf4j/jcl.jar:/usr/share/java/simple-core.jar:/usr/share/java/commons-dbcp.jar:/usr/share/java/commons-pool.jar:/usr/share/java/quartz-oracle.jar:/usr/share/java/postgresql-jdbc.jar:/usr/share/java/mchange-commons.jar:/usr/share/java/javassist.jar -Dwrapper.key=olCO7PmbUVUZ_hnn -Dwrapper.port=32001 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.pid=6488 -Dwrapper.version=3.2.3 -Dwrapper.native_library=wrapper -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 com.redhat.rhn.taskomatic.core.TaskomaticDaemon

If I look at the following file: /usr/share/rhn/config-defaults/rhn_taskomatic_daemon.conf, I see the following values:

wrapper.java.initmemory=256

wrapper.java.maxmemory=1024

However there haven't been any obvious error messages. Would it be reasonable to do a Java Heap Dump to see if I should do anything to update/troubleshoot?

I've also found these two threads:

https://access.redhat.com/solutions/43122

https://access.redhat.com/discussions/1218823

https://rhn.redhat.com/errata/RHBA-2014-1651.html#Red%20Hat%20Satellite%20%28v.%205.6%20for%20RHEL%205%29

I checked the version of spacewalk-java

[root@foo rhn]# yum info spacewalk-java
Loaded plugins: product-id, rhnplugin, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
Installed Packages
Name       : spacewalk-java
Arch       : noarch
Version    : 2.0.2
Release    : 46.el5sat
Size       : 12 M
Repo       : installed
Summary    : Java web application files for Spacewalk
URL        : https://fedorahosted.org/spacewalk
License    : GPLv2
Description: This package contains the code for the Java version of the Spacewalk Web Site.

I opened a ticket with support.

It was as simple as the following which are in a cronjob

rm -rf /var/cache/yum/*
yum clean all
yum repolist

Once running this, the clients are seeing the updates from the RH Satellite.

thanks

Glad you got it resolved. Thanks for sharing the solution!

Where did you run this command?

a side note, somewhat related, see this discussion https://access.redhat.com/discussions/1597433, and the new base channels are available for RHEL 6.7 for Satellite 5.x. NOTE: that discussion has to do with -disconnected- satellite servers. Just an FYI.

Each client has this cronjob on it which runs daily.

I wouldn't recommend that. Especially for larger number of clients. If your clients work with outdated repository metadata and do not update them, isn't your problem caused by having metadata_expire or a similar option set to a high number? Or, wouldn't setting of metadata_expire to 1 day solve your problem exactly as your current cron job does?

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.