Gluster Client Cannot Connect to Brick due to Gluster Reporting Incorrect Port (Multiplex)

Solution In Progress - Updated -

Issue

After upgrading Gluster, the heal info count never becomes zero, and logs show client trying to connect to a port that is different from the port Gluster is reporting for the brick:

Client logs say client is connecting to port 49155 (for example):

[2017-06-01 08:56:05.149017] I [rpc-clnt.c:2001:rpc_clnt_reconfig] 0-data-client-1: changing port to 49155 (from 0)
[2017-06-01 08:56:05.152941] E [socket.c:2318:socket_connect_finish] 0-data-client-1: connection to X.X.X.X:49155 failed (Connection refused); disconnecting socket
[2017-06-01 08:56:09.153591] I [rpc-clnt.c:2001:rpc_clnt_reconfig] 0-data-client-1: changing port to 49155 (from 0)
[2017-06-01 08:56:09.157781] E [socket.c:2318:socket_connect_finish] 0-data-client-1: connection to X.X.X.X:49155 failed (Connection refused); disconnecting socket
[2017-06-01 08:56:13.158146] I [rpc-clnt.c:2001:rpc_clnt_reconfig] 0-data-client-1: changing port to 49155 (from 0)
[2017-06-01 08:56:13.162099] E [socket.c:2318:socket_connect_finish] 0-data-client-1: connection to X.X.X.X:49155 failed (Connection refused); disconnecting socket
[2017-06-01 08:56:17.162637] I [rpc-clnt.c:2001:rpc_clnt_reconfig] 0-data-client-1: changing port to 49155 (from 0)
[2017-06-01 08:56:17.166604] E [socket.c:2318:socket_connect_finish] 0-data-client-1: connection to X.X.X.X:49155 failed (Connection refused); disconnecting socket
[2017-06-01 08:56:21.167186] I [rpc-clnt.c:2001:rpc_clnt_reconfig] 0-data-client-1: changing port to 49155 (from 0)
[2017-06-01 08:56:21.171153] E [socket.c:2318:socket_connect_finish] 0-data-client-1: connection to X.X.X.X:49155 failed (Connection refused); disconnecting socket
[2017-06-01 08:56:25.172089] I [rpc-clnt.c:2001:rpc_clnt_reconfig] 0-data-client-1: changing port to 49155 (from 0)
[2017-06-01 08:56:25.176367] E [socket.c:2318:socket_connect_finish] 0-data-client-1: connection to X.X.X.X:49155 failed (Connection refused); disconnecting

However, the brick appears to be listening on port 49152:

# ps ax | grep -i brick | grep data
root     22436  0.5  0.0 1217564 26164 ?       Ssl  12:31   0:40 /usr/sbin/glusterfsd -s X.X.X.X --volfile-id data.X.X.X.X.gluster_bricks-data-data -p /var/lib/glusterd/vols/data/run/X.X.X.X-gluster_bricks-data-data.pid -S /var/run/gluster/77a68376a313c246358b3d10487b0e94.socket --brick-name /gluster_bricks/data/data -l /var/log/glusterfs/bricks/gluster_bricks-data-data.log --xlator-option *-posix.glusterd-uuid=f889a05f-648e-4890-b1e6-637e33b69803 --brick-port 49152 --xlator-option data-server.listen-port=49152

Environment

  • Red Hat Gluster Storage 3.3

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In