Gluster rebalance failed with no subvolume up errors
Issue
[2017-07-19 20:21:55.897695] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-xxx-client-6: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2017-07-19 20:21:55.897737] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-xxx-client-6: disconnected from xxx-client-6. Client process will keep trying to connect to glusterd until brick's port is available
[2017-07-19 20:21:55.902838] I [MSGID: 108006] [afr-common.c:4131:afr_local_init] 0-xxx-replicate-5: no subvolumes up
[2017-07-19 20:21:55.940539] I [MSGID: 109063] [dht-layout.c:702:dht_layout_normalize] 0-xxx-dht: Found anomalies in / (gfid = 00000000-0000-0000-0000-000000000001). Holes=1 overlaps=0
[2017-07-19 20:21:55.940565] W [MSGID: 109005] [dht-selfheal.c:1782:dht_selfheal_directory] 0-xxx-dht: Directory selfheal failed: 1 subvolumes down.Not fixing. path = /, gfid = 00000000-0000-0000-0000-000000000001
[2017-07-19 20:21:55.940635] I [dht-rebalance.c:2651:gf_defrag_start_crawl] 0-xxx-dht: gf_defrag_start_crawl using commit hash 0
[2017-07-19 20:21:55.940674] I [MSGID: 108006] [afr-common.c:4131:afr_local_init] 0-xxx-replicate-5: no subvolumes up
[2017-07-19 20:21:56.021657] I [MSGID: 109081] [dht-common.c:3819:dht_setxattr] 0-xxx-dht: fixing the layout of /
[2017-07-19 20:21:56.021679] W [MSGID: 109016] [dht-selfheal.c:1474:dht_fix_layout_of_directory] 0-xxx-dht: Layout fix failed: 1 subvolume(s) are down. Skipping fix layout.
[2017-07-19 20:21:56.021906] E [MSGID: 109026] [dht-rebalance.c:2693:gf_defrag_start_crawl] 0-xxx-dht: fix layout on / failed
[2017-07-19 20:21:56.021930] I [MSGID: 109028] [dht-rebalance.c:3039:gf_defrag_status_get] 0-xxx-dht: Rebalance is failed. Time taken is 1.00 secs
[2017-07-19 20:21:56.021936] I [MSGID: 109028] [dht-rebalance.c:3043:gf_defrag_status_get] 0-xxx-dht: Files migrated: 0, size: 0, lookups: 0, failures: 1, skipped: 0
[2017-07-19 20:21:56.022077] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7dc5) [0x7face572adc5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x55ba9798c785] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x55ba9798c609] ) 0-: received signum (15), shutting down
Environment
glusterfs 3.*
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
