Unable to use push functionality when a server registers to the Satellite server through the Proxy
Environment
- Red Hat Satellite
- Red Hat Satellite Proxy
- osad/push functionality
Issue
- I am unable to use push functionality when a server registers to the satellite server through the proxy. If the server registers directly to the satellite, push functionality works.
- Seeing 'No route to host' or 'Connection refused' errors in /var/log/messages on the Satellite when osad clients connect to the Proxy.
- Or not seeing the Proxy initiate any s2s connection to the Satellite when an osad client connects to the Proxy (and an s2s connection doesn't already exist).
Resolution
- Make sure port 5269 is accessible on both the Satellite and the Proxy.
- Another solution is to try deleting the jabberd db files on the Proxy:
# rm -rf /var/lib/jabberd/db/*
# service jabberd restart
Root Cause
- iptables rules blocking port 5269 on the Proxy meant that the Satellite wasn't able to connect to the s2s process running on the Proxy.
-
Port 5269 is what the jabberd s2s process listens on for messages between the Satellite and the Proxy.
-
In another case, the Proxy simply wasn't initiating an s2s connection between the Proxy and Satellite when an osad client first connected to the Proxy. Normally when the first osad client connects to the Proxy, the Proxy then initiates an s2s connection to the Satellite to allow jabberd messages to be sent between the Proxy and the Satellite.
- The root cause of this was that the version of jabberd installed on the Proxy was relatively old and thus wasn't removing the /var/lib/jabberd/db/* files whenever the jabberd service started. The solution was to manually delete these files. Updating the jabberd package to the latest version meant that the /var/lib/jabberd/db/* files were automatically deleted each time jabberd started.
Diagnostic Steps
- /var/log/messages on the Satellite:
Jul 7 16:53:10 satellite jabberd/s2s[2928]: [8] [::ffff:10.64.24.190, port=58111] incoming connection
Jul 7 16:53:10 satellite jabberd/s2s[2928]: [8] [::ffff:10.64.24.190, port=58111] incoming stream online (id 3uemjcfiz76w68x7giibsz5vq0z6frhilw558ko8)
Jul 7 16:53:10 satellite jabberd/s2s[2928]: [8] [::ffff:10.64.24.190, port=58111] received dialback auth request for route 'satellite.example.com/proxy.example.com'
Jul 7 16:53:14 satellite jabberd/s2s[2928]: dns lookup for proxy returned 1 result (ttl 86400)
Jul 7 16:53:14 satellite jabberd/s2s[2928]: [9] [10.64.24.190, port=5269] outgoing connection for 'proxy.example.com'
Jul 7 16:53:14 satellite jabberd/s2s[2928]: [9] [10.64.24.190, port=5269] write error: No route to host (113)
Jul 7 16:53:14 satellite jabberd/s2s[2928]: [9] [10.64.24.190, port=5269] disconnect, packets: 0
- /var/log/messages on the Proxy:
Jul 7 16:53:08 proxy jabberd/c2s[1733]: [7] requesting session: jid=osad-9c1dc6fb51@proxy.example.com/osad
Jul 7 16:53:08 proxy jabberd/sm[1725]: session started: jid=osad-9c1dc6fb51@proxy.example.com/osad
Jul 7 16:53:10 proxy jabberd/s2s[1741]: dns lookup for satellite returned 1 result (ttl 86400)
Jul 7 16:53:10 proxy jabberd/s2s[1741]: [8] [10.64.9.111, port=5269] outgoing connection for 'satellite.example.com'
Jul 7 16:53:10 proxy jabberd/s2s[1741]: [8] [10.64.9.111, port=5269] sending dialback auth request for route 'proxy.example.com/satellite.example.com'
Jul 7 16:54:14 proxy jabberd/s2s[1741]: [8] [10.64.9.111, port=5269] error: Stream error (dialback timed out)
Jul 7 16:54:14 proxy jabberd/s2s[1741]: [8] [10.64.9.111, port=5269] disconnect, packets: 0
- It can be seen that the Satellite is getting a 'No route to host (113)' error (or a 'Connection Refused (111)') error when trying to connect to port 5269 on the Proxy. And on the Proxy it can be seen the s2s dialback from Proxy to Satellite is timing out - 'Stream error (dialback timed out)'.
- The Satellite has ports 5222 (c2s) and 5269 (s2s) open for incoming connections:
$ egrep '5222|5269' /etc/sysconfig/iptables
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 5222 -j ACCEPT
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 5269 -j ACCEPT
- However the Proxy only has port 5222 open:
$ egrep '5222|5269' /etc/sysconfig/iptables
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 5222 -j ACCEPT
- In another case, the Proxy simply wasn't initiating an s2s connection between the Proxy and Satellite. Here is the osad client connecting to the Proxy:
Jul 14 16:21:59 proxy.example.com jabberd/c2s[13031]: [7] [172.24.97.183, port=51984] connect
Jul 14 16:21:59 proxy.example.com jabberd/c2s[13031]: [7] legacy authentication succeeded: host=, username=osad-4f7c1df02d, resource=osad, TLS negotiated
Jul 14 16:21:59 proxy.example.com jabberd/c2s[13031]: [7] requesting session: jid=osad-4f7c1df02d@proxy.example.com/osad
Jul 14 16:21:59 proxy.example.com jabberd/sm[13024]: session started: jid=osad-4f7c1df02d@proxy.example.com/osad
... but for some reason the s2s connection to the Satellite isn't being initiated. The next line after this should the be the 'dns lookup for ...' message. After the client connects to c2s on the Proxy, and there is no s2s connection between the Satellite and Proxy, then one should be automatically established. But it isn't happening here.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
