Red Hat Satellite Capsule sync failing with an error "xxxx is unreachable. 500 Internal Server Error"

Solution Verified - Updated -

Environment

  • Red Hat Satellite 6.9
  • Red Hat Satellite Capsule 6.9

Issue

  • Sync task is getting paused with the following error:

     xxxxxxxxxxxxxxx is unreachable. 500 Internal Server Error (Katello::Errors::CapsuleCannotBeReached)
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.137/app/lib/katello/capsule_content.rb:147:in `rescue in ping_pulp'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.137/app/lib/katello/capsule_content.rb:145:in `ping_pulp'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.137/app/lib/actions/katello/capsule_content/sync.rb:14:in `plan'

  • Red Hat Satellite Capsule sync is not working after upgrade

Resolution

  • Run the below commands on Red Hat Satellite Capsule server:

     # for i in pulp_celerybeat pulp_resource_manager pulp_workers pulp_streamer; do service $i stop; done
 # sudo -u apache pulp-manage-db
 # foreman-maintain service restart

For more KB articles/solutions related to Red Hat Satellite 6.x Capsule Sync Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Capsule Sync Issues

Root Cause

  • The database has not been migrated to the current version.

Diagnostic Steps

  • katello-service status showing following message:

    [root@example]#katello-service status
: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)   The database has not been migrated to the current version. Run                                                    pulp-manage-db and restart the application.
Aug 03 12:10:46 pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560) Traceback (most recent call last):
Aug 03 12:10:46 pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/                                                   application.py", line 111, in wsgi_application
Aug 03 12:10:46  pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)     _initialize_web_services()
Aug 03 12:10:46  pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/                                                   application.py", line 74, in _initialize_web_services
Aug 03 12:10:46  pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)     migration_models.check_package_versions()
Aug 03 12:10:46  pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)   File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/m                                                   odels.py", line 310, in check_package_versions
Aug 03 12:10:46 pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560)     raise Exception(error_message)
Aug 03 12:10:46  pulp[24555]: pulp.server.webservices.application:ERR                                                   OR: (24555-90560) InitializationException: The database has not been migrated to th                                                   e current version. Run pulp-manage-db and restart the application.
Aug 03 12:10:46  pulp[24555]: pulp.server.webservices.application:CRI                                                   TICAL:

  • Below error message is observed in '/var/log/messages' file on Red Hat Satellite Capsule server

    May 11 08:02:54 capsule qdrouterd: Mon May 11 08:02:54 2015 ERROR (error) Invalid message: Insufficient Data to Determine Tag
May 11 08:02:54 capsule qdrouterd: Mon May 11 08:02:54 2015 ERROR (error) Invalid message: Can't convert message field body
May 11 08:02:54 capsule qdrouterd: Mon May 11 08:02:54 2015 ROUTER (error) Control message error: opcode=HELLO body=None
May 11 08:02:54 capsule qdrouterd: Traceback (most recent call last):
May 11 08:02:54 capsule qdrouterd: File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/router/engine.py", line 135, in handleControlMessage
May 11 08:02:54 capsule qdrouterd: self.hello_protocol.handle_hello(msg, now, link_id)
May 11 08:02:54 capsule qdrouterd: File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/router/hello.py", line 50, in handle_hello
May 11 08:02:54 capsule qdrouterd: if msg.is_seen(self.id):
May 11 08:02:54 capsule qdrouterd: File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/router/data.py", line 129, in is_seen
May 11 08:02:54 capsule qdrouterd: return self.seen_peers.count(_id) > 0
May 11 08:02:54 capsule qdrouterd: AttributeError: 'NoneType' object has no attribute 'count'

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments