In an MRG-M cluster, when killing an application broker (AB), the messages log for qpidd indicates repeated 127.0.0.1 connections between the clustered application brokers which are then rejected.

Solution Verified - Updated -

Environment

  • Red Hat Enterprise MRG Messaging (MRG-M) 2.3

Issue

  • When killing an application broker (AB), the messages log for qpidd indicates repeated 127.0.0.1 connections between the clustered application brokers which are then rejected.

Resolution

The backup brokers had its own hostname as part of the list of address used for fail over which is why you see these messages. This message is harmless.

Diagnostic Steps

In cases of killing an application broker (AB) we see messages in the logs indicating repeated connections between the clustered application brokers which are then rejected.
The sequences look like:

Apr 1 07:30:16 system0 qpidd[31640]: 2013-04-01 07:30:16 [Network] info Set TCP_NODELAY on connection to 127.0.0.1:58404
Apr 1 07:30:16 system0 qpidd[31640]: 2013-04-01 07:30:16 [Model] debug Create connection. user:anonymous rhost:127.0.0.1:10000-127.0.0.1:58404
Apr 1 07:30:16 system0 qpidd[31640]: 2013-04-01 07:30:16 [HA] debug Backup: Rejected connection 127.0.0.1:10000-127.0.0.1:58404
Apr 1 07:30:16 system0 qpidd[31640]: 2013-04-01 07:30:16 [Model] debug Delete connection. user:anonymous rhost:127.0.0.1:10000-127.0.0.1:58404

Which is odd, since it's a localhost connection. Most are from other nodes in the cluster:

Apr 1 07:30:17 system0 qpidd[31640]: 2013-04-01 07:30:17 [Network] info Set TCP_NODELAY on connection to 10.0.0.31:48614
Apr 1 07:30:17 system0 qpidd[31640]: 2013-04-01 07:30:17 [Model] debug Create connection. user:anonymous rhost:10.0.0.30:10000-10.0.0.31:48614
Apr 1 07:30:17 system0 qpidd[31640]: 2013-04-01 07:30:17 [HA] debug Backup: Rejected connection 10.0.0.30:10000-10.0.0.31:48614
Apr 1 07:30:17 system0 qpidd[31640]: 2013-04-01 07:30:17 [Model] debug Delete connection. user:anonymous rhost:10.0.0.30:10000-10.0.0.31:48614

This same cluster is also running other brokers and those don't exhibit this when connecting to each other.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments