Yum has issues accessing the metadata of a channel on Red Hat Satellite or cannot find updated packages

Solution Verified - Updated -

Environment

  • Red Hat Satellite 5

Issue

  • yum cannot retrieve metadata from the satellite:

    # yum check-update 
    Loaded plugins: rhnplugin, security 
    Error: Cannot retrieve repository metadata (repomd.xml) for repository: rhel-i386-server-5.
    Please verify its path and try again 
    
  • Client system shows 0 packages in base channel:

    # yum repolist
    Loaded plugins: rhnplugin, security
    repo id               repo name                                             status
    rhel-s390x-server-5   Red Hat Enterprise Linux (v. 5 64-bit IBM System z)   enabled:  0
    
  • yum update on the client fails with Missing Dependency errors for already-installed packages:

    Error: Missing Dependency: libpango-1.0.so.0 is needed by package sound-juicer-2.16.0-3.el5.i386 (installed)
    Error: Missing Dependency: libpango-1.0.so.0 is needed by package vte-0.14.0-2.el5.i386 (installed)
    Error: Missing Dependency: libpangocairo-1.0.so.0 is needed by package gnome-keyring-0.6.0-1.fc6.i386 (installed)
    
  • client fails to list the latest package available in Satellite

Resolution

  • Verify that satellite-sync has completed successfully for the channel.
  • Make sure that the Taskomatic service starts successfully on the Satellite server:
# /etc/init.d/taskomatic restart
  • Monitor /var/log/rhn/rhn_taskomatic_daemon.log for any errors during startup.
  • On the client side, check if yum rhnplugin enabled in yum configuration:
# grep plugins /etc/yum.conf 
plugins=1
# grep ^enabled /etc/yum/pluginconf.d/rhnplugin.conf
enabled = 1
  • If the above steps do not resolve the issue, then clean the satellite and client cache files manually:
On the Satellite
  • Clean the channel cache:
    • For Satellite 5.3 and later (the channel label can be found by viewing the channel details page in the Satellite web interface):
Move repodata to a backup location,

# mv /var/cache/rhn/repodata/<channel-label> /some/backup/directory/
  • For Satellite 5.2 and earlier (the channel id can be found by viewing the value of "cid" in the URL when viewing the channel details page in the Satellite web interface):
Move repodata to a backup location,

# mv  /var/cache/rhn/repomd*-<channel-id> /some/backup/directory/
  • Restart taskomatic:
#  /etc/init.d/taskomatic restart
On the client
  • Clean cache on the client:
# yum clean all
  • Force the Satellite to start generating the channel cache files, by running a yum command from the client:
# yum check-update
On the Satellite
  • Check if the previously-deleted files are being created, and be patient: the regeneration process can take upwards of 30 minutes for large base channels.
  • If, once the files have been regenerated, "yum update" on the client still yields "Missing Dependency" errors, and the Satellite server is on version 5.4 or later, then repeat the above steps after first deleting the repodata information stored in the Oracle database:
# service taskomatic stop
# python regen-repodata.py -c <channel-label> --cleandb --force --db --url https://satellite.example.com/rpc/api

# service taskomatic start

The regen-repodata.py script is attached to this kbase and available on github

Information on the usage of regen-repodata are available through the usage of

# python regen-repodata.py -h

Root Cause

  • For system to be able to receive updates from Satellite, yum rhnplugin have to be active.
  • On Red Hat Network Satellite version 5.2 and earlier, the repomd.xml files of a channel will only be generated after a client makes a request for that particular channel, so if a channel is updated and the clients request the data they will keep getting an error until the repomd.xml files are ready.
  • On Red Hat Network Satellite version 5.3 and later, the taskomatic service will automatically regenerate channel cache when a channel has been created or updated, or if a client make a request for a channel that currently has no cache. If a client makes a request for the repomd.xml file while new cache files are still being generated, then the old repomd.xml file (if it exists) will be delivered to the client, and no error will be reported.

Attachments

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

16 Comments

Under Satellite 5.3, is there a way to trigger Taskomatic's rebuilding of the cached repodata?  I followed these steps and then waited 2 hrs before repodata was there.


Please note that if you are importing channels from a Satellite 5\,2 or older you have to set the checksum field to SHA1 otherwise it will not generate the data, you may see errors like the following in that case:


\[DefaultQuartzScheduler\_Worker\-7\] INFO
com\,redhat\,rhn\,taskomatic\,task\,ChannelRepodata \- Creating ChannelRepodataWorker
with prefixPath\(rhn/repodata\), mountPoint\(/var/cache\) for channel\_label
\(nh\-i386\-client\-supplementary\-5\)
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,391 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- File Modified
Date:1970\-01\-01 01:00:00 CET
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,391 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- Channel Modified
Date:2011\-03\-03 16:03:46 CET
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,425 \[Thread\-62\]
WARN com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- No repo will be
generated for channel nh\-i386\-client\-supplementary\-5
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,425 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- Removing
nh\-i386\-client\-supplementary\-5
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,426 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- Couldn't remove
/var/cache/rhn/repodata/nh\-i386\-client\-supplementary\-5/primary\,xml\,gz
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,426 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- Couldn't remove
/var/cache/rhn/repodata/nh\-i386\-client\-supplementary\-5/filelists\,xml\,gz
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,426 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- Couldn't remove
/var/cache/rhn/repodata/nh\-i386\-client\-supplementary\-5/other\,xml\,gz
INFO | jvm 1 | 2011/03/04 12:03:00 | 2011\-03\-04 12:03:00,426 \[Thread\-62\]
INFO com\,redhat\,rhn\,taskomatic\,task\,repomd\,RepositoryWriter \- Couldn't remove
/var/cache/rhn/repodata/nh\-i386\-client\-supplementary\-5/repomd\,xml

removing sql as per previous sbr talks about removing all sql from published kbases ; this script does the same job but doesn't touch the db.

Is there a fix for this that is coming?

I tried running this script on satellite 5.6 and got this error

[root@satellite sat-config]# service taskomatic stop
Stopping RHN Taskomatic...
Stopped RHN Taskomatic.
[root@satellite sat-config]# python regen-repodata.py -c rhel-x86_64-server-6 --cleandb --db --url https://satellite.cloud.net/rpc/api
Login details for https://satellite.cloud.net/rpc/api

Login: userredact
Password:

satellite version 5.6.0
Traceback (most recent call last):
File "regen-repodata.py", line 276, in
main(version)
File "regen-repodata.py", line 262, in main
regen_channel_db(key,[options.channel],options.clean_db)
File "regen-repodata.py", line 217, in regen_channel_db
h.execute(channel=label)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 163, in execute
return apply(self.execute_wrapper, (self._execute, ) + p, kw)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 273, in _execute_wrapper
retval = apply(function, p, kw)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 217, in _execute
return self._execute
(args, kwargs)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 287, in execute
self._real_cursor.execute(self.sql, params)
psycopg2.ProgrammingError: missing FROM-clause entry for table "rhn_repo_regen_queue_id_seq"
LINE 1: ...NEL_LABEL, REASON, BYPASS_FILTERS, FORCE) VALUES (rhn_repo_r...
^

Any Ideas on how to resolve ?

update 3.2.1 of the script fixes that issue.

We tried this, and the python script version 3.2.1 worked...

We checked the taskomatic logs and it seems java is tripping over itself. Next stop, diagnose the java issues...

I had a problem with a java error in the taskomatic logs, not enough memory allocated to java, here's the article I found to fix it:
https://access.redhat.com/solutions/43122

Not enough memory allocated to java in the taskomatic daemon config, and java heap itself needed to be increased.

I was getting this error with Satellite 5.6

It took a combination of 3 articles to fix it for me:
https://access.redhat.com/solutions/62078
https://access.redhat.com/solutions/61964
https://access.redhat.com/solutions/19303

To regenerate the repo data on RHN Satellite 5.5, this did it for me.

rm -rf /var/cache/rhn/repodata/insert-channel-id/*
service taskomatic stop
regen-repodata.py -c insert-channel-id --cleandb --force --db --url https://localhost/rpc/api
service taskomatic start

Please note that just removing the repo data did not automatically regenerate when just restarting taskomatic after having rm:d the data.

Part of the problem in my case, a rhel 6 satellite version 5.6, I found that spacewalk-java (and it's other rpms) had not been updated since the initial installation of the satellite server, even though I faithfully download/ingest and deploy satellite base and incremental channels as they are distributed per our patch plan for our Red Hat products.

The funny thing is, I keep up to date with all patches available (key word, available) that I download in my base and incremental channels. Since none of my four satellite servers are facing the public internet and getting patched as a "connected" satellite, I did not receive the 13 subsequent updates of spacewalk-java.

Once I manually downloaded the four related rpms for spacewalk-java (not part of any current downloadable iso channel), my repos would create again. I did also update the rpm, and increased the memory for java and tomcat per that Red Hat solution. Another customer I support had this issue on a rhel 5 satellite v5.5. More details in this discussion I started on this matter.

Tried script 3.2.2 .. no luck
[root@svr001 infra-rhel-x86_64-server-6]# python /home/kzwart/regen-repodata.py -c infra-rhel-x86_64-server-6 --cleandb --force --db --url https://svr001.mgt.rad.lan/rpc/api
Login details for https://svr001.mgt.rad.lan/rpc/api

Login: kzwart
Password:

satellite version 5.6.0
Traceback (most recent call last):
File "/home/kzwart/regen-repodata.py", line 268, in
main(version)
File "/home/kzwart/regen-repodata.py", line 254, in main
regen_channel_db(key,[options.channel],options.clean_db)
File "/home/kzwart/regen-repodata.py", line 209, in regen_channel_db
h.execute(channel=label)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 163, in execute
return apply(self.execute_wrapper, (self._execute, ) + p, kw)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 273, in _execute_wrapper
retval = apply(function, p, kw)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 217, in _execute
return self._execute
(args, kwargs)
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 287, in execute
self._real_cursor.execute(self.sql, params)
psycopg2.ProgrammingError: missing FROM-clause entry for table "rhn_repo_regen_queue_id_seq"
LINE 1: ...NEL_LABEL, REASON, BYPASS_FILTERS, FORCE) VALUES (rhn_repo_r...
^
How to proceed?. The repomd is regenerated. Memory is ok. But still dependancy errors

Hi Kees,

Try using the latest one from the dev branch on github (https://github.com/FDewaleyne/regen-repodata/tree/dev)

That one worked for me (at least you will get rid of the errors in the script, did not check the yum errors yet) on Satellite 5.6

Had the same problems with "Cannot retrieve repository metadata (repomd.xml) for repository".
It turned out that we in our attempts to secure our systems had changed the SSL settings (removed SSLv3) on the satellite server which caused the metadata transfer to fail.
After a bit of experimenting with different protocol and cipher settings, the only working combo I found was
SSLProtocol all -SSLv2
SSLCipherSuite HIGH:-SSLv2

If anyone knows how to run a satellite 5.6 without SSLv3 it would be greatly appreciated.
Note! If I disable sslv3 the satellite server + webgui works, but I'm no longer able to update my client servers after running a 'yum clean all'.

Hi.

I upgraded from RHN 5.4 to RHN 5.6 and now any new registered client reports the following

# yum check-update
Loaded plugins: rhnplugin, security
This system is receiving updates from RHN Classic or RHN Satellite.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: rhel-x86_64-server-5. Please verify its path and try again

This error messages also appears in old registered clients if you run "yum clean all" and try to check/list/install updates "yum list updates".

I have followed this procedure and have had to run regen-repodata.py (4.0.2 release downloaded from GITHUB) but it does not seem to recognize the channel name:

# python regen-repodata.py -c rhel-x86_64-server-5 --cleandb --force --db --url https://mysatellite.qfdn/rpc/api
channel x does not have a checksum
channel e does not have a checksum
channel h does not have a checksum
channel v does not have a checksum
channel - does not have a checksum
channel l does not have a checksum
channel s does not have a checksum
channel r does not have a checksum
channel 5 does not have a checksum
channel 4 does not have a checksum
channel 6 does not have a checksum
channel 8 does not have a checksum
channel _ does not have a checksum
Finished queueing the new jobs into the database
Restart taskomatic and schedule the 'channel-repodata-bunch' task once to continue
Clear the needed cache if your systems display incorrect update counts afterwards (option --cleancache)

By the way, no user or password are asked.

I'm sure the channel name is correct because I listed the channels:

satellite version 5.6.0
Channels:
                                       Label |   Checksum | Name
                              rhel-i386-as-4 |            | Red Hat Enterprise Linux AS (v. 4 for 32-bit x86)
                              rhel-i386-es-4 |            | Red Hat Enterprise Linux ES (v. 4 for 32-bit x86)
                            rhel-x86_64-es-4 |            | Red Hat Enterprise Linux ES (v. 4 for AMD64/Intel EM64T)
                        rhel-x86_64-server-6 |     sha256 | Red Hat Enterprise Linux Server (v. 6 for 64-bit x86_64)
                          rhel-i386-server-5 |       sha1 | Red Hat Enterprise Linux (v. 5 for 32-bit x86)
                        rhel-x86_64-server-5 |       sha1 | Red Hat Enterprise Linux (v. 5 for 64-bit x86_64)
    redhat-rhn-satellite-5.4-server-x86_64-5 |       sha1 | Red Hat Network Satellite (v5.4 for Server v5 AMD64 / Intel64)
    redhat-rhn-satellite-5.5-server-x86_64-5 |       sha1 | Red Hat Network Satellite (v5.5 for Server v5 AMD64 / Intel64)
                rhn-tools-rhel-i386-server-5 |       sha1 | Red Hat Network Tools for RHEL Server (v.5 32-bit x86)
              rhn-tools-rhel-x86_64-server-5 |       sha1 | Red Hat Network Tools for RHEL Server (v.5 64-bit x86_64)
                             rhel5_x64_local |       sha1 | rhel5_x64_local
                rhel-x86_64-server-cluster-5 |       sha1 | RHEL Clustering (v. 5 for 64-bit x86_64)
                     rhel-x86_64-server-ha-6 |     sha256 | RHEL Server High Availability (v. 6 for 64-bit x86_64)
                     rhel-x86_64-server-lb-6 |     sha256 | RHEL Server Load Balancer (v. 6 for 64-bit x86_64)
              rhn-tools-rhel-x86_64-server-6 |     sha256 | RHN Tools for RHEL (v. 6 for 64-bit x86_64)

Finally, I downloaded python program release 3.2.2 and it runs fine. It has queued all channels.