satellite v5.5 - sync stuck at "Freeing read locks"

Latest response

Hi Everyone,

 

I am playing around with satellite v5.5....it's a freshly installed system and I keep getting stuck at the message "Freeing read locks" (as shown below).  So far, I have let it sit for approx 8hrs with no progress.  Once taskomatic died for no reason but upon restart there was no changes...still stuck.  So, I am just curious if others have experienced this behavior since I do not recall seeing this in earlier versions of satellite/spacewalk.  

 

Below I have listed some of the things that I have tried...all with no luck unfortunately.

1. multiple restarts of satellite service and reboots then resyncing
2. removing the channel with "spacewalk-remove-channel -c rhel-x86_64-server-6" and resyncing
3. the standard cache clearing trick "rm -rf /var/cache/rhn/satsync/*"
4. yum-complete-transaction  -  not even sure if this applies here but I found it on some forum and tried it....no luck
5. different options with satellite-sync such as --force-all-packages

 

<<<<Beginning of output from the sync>>>>>

 

14:23:57 Red Hat Network Satellite - live synchronization
14:23:57    url: https://satellite.rhn.redhat.com
14:23:57    debug/output level: 1
14:23:58    db:  rhnsat/<password>@//localhost:1521/rhnsat.world
14:23:58
14:23:58 Retrieving / parsing channel-families data
14:24:06 channel-families data complete
14:24:06
14:24:06 Retrieving / parsing product names data
14:24:07 product names data complete
14:24:14
14:24:14 Retrieving / parsing arches data
14:24:15 arches data complete
14:24:15
14:24:15 Retrieving / parsing additional arches data
14:24:16 additional arches data complete
14:24:16
14:24:16 Retrieving / parsing channel data
14:25:14    p = previously imported/synced channel
14:25:14    . = channel not yet imported/synced
14:25:14    base-channels:
14:25:14       p rhel-x86_64-server-6                     8654
14:25:14
14:25:16 Channel data complete
14:25:16
14:25:16 Retrieving short package metadata (used for indexing)
14:25:16    Retrieving / parsing short package metadata: rhel-x86_64-server-6 (8654)
14:25:21 Diffing package metadata (what's missing locally?): rhel-x86_64-server-6
            ________________________________________
Diffing:    ######################################## - complete
14:29:18
14:29:18 Downloading package metadata
14:29:18    Retrieving / parsing *relevant* package metadata: rhel-x86_64-server-6 (2900)
14:29:18    * WARNING: this may be a slow process.
            ________________________________________
Downloading:######################################## - complete
14:38:04
14:38:04 Downloading rpm packages
14:38:04    Fetching any missing RPMs: rhel-x86_64-server-6 (2900)
14:39:03    Total size: 5.02 GiB
14:44:31       Downloaded 5.01 GiB of 5.02 GiB. Estimated remaining time: 5:17:32
14:44:31     2896/2900 Fetch successful: fence-agents-3.1.5-17.1.el6_3.x86_64.rpm (153920 bytes)
14:44:34       Downloaded 5.02 GiB of 5.02 GiB. Estimated remaining time: 0:11:47
14:44:34     2897/2900 Fetch successful: selinux-policy-targeted-3.7.19-155.el6_3.6.noarch.rpm (2748184 bytes)
14:44:37       Downloaded 5.02 GiB of 5.02 GiB. Estimated remaining time: 0:06:27
14:44:37     2898/2900 Fetch successful: selinux-policy-3.7.19-155.el6_3.6.noarch.rpm (1314680 bytes)
14:44:37       Downloaded 5.02 GiB of 5.02 GiB. Estimated remaining time: 0:02:00
14:44:37     2899/2900 Fetch successful: selinux-policy-minimum-3.7.19-155.el6_3.6.noarch.rpm (2485932 bytes)
14:44:40       Downloaded 5.02 GiB of 5.02 GiB. Estimated remaining time: 0:00:00
14:44:40     2900/2900 Fetch successful: selinux-policy-mls-3.7.19-155.el6_3.6.noarch.rpm (2419448 bytes)
14:44:40 Processing rpm packages complete
14:44:40
14:44:40 Importing package metadata
14:44:40    Importing *relevant* package metadata: rhel-x86_64-server-6 (2900)
            ________________________________________
Importing:  Freeing read locks for locker 0x10: 25537/140379561912064
Freeing read locks for locker 0x12: 25537/140379561912064

 

Thanks,

 

Nick Greene

Responses

Hi Nick,

 

I haven't see such an error on satellite-sync. If it is always failing at "Importing package metadata" and with the same error then the first thing that I would recommend to check the underlying rpm db.

  • Are you able to install/update/remove a package on the satellite ?
  • Does `rpm -qa` lists the installed packages correctly ?

We might want to ensure the rpm db is not in a bad state and if it is then consider rebuilding the rpm db referring to the knowledge solution - How to rebuild my RPM database?

 

Additionally we can also increase the debug level in /etc/rhn/rhn.conf by setting/adding

debug=6

execute satellite-sync and check if we see any additional details on the console as well as /var/log/rhn/rhn_server_satellite.log

 

Paresh

Thanks for the reply/suggestions.  Based on only 1 reply, apparently, not too many people are having this issue.  But here is what worked for me.  Perhaps, someone else might find it useful.

 

Looking in /var/lib/rpm I noticed that all of my systems  _db*  files were missing for the rpm database.  However, I was still able to run rpm commands successfully like "rpm -qa" and I even installed a test rpm with no issues.  I then ran "rpm -vv --rebuilddb" and still no luck.  Next I ran "rpm --initdb" and finally the _db* files were created.  I then ran the sync and it worked.

 

Nick Greene.

Nick, thanks for following up with your solution here. Hopefully it will help out anyone who runs into this down the line.

Thank You Nick for sharing steps which worked for you. Based on it, I have created a knowledge solution Why does satellite-sync keeps getting stuck at the message "Freeing read locks for locker ..." ?