Gluster rebalance failed with no subvolume up errors

Solution In Progress - Updated -

Issue

[2017-07-19 20:21:55.897695] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-xxx-client-6: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2017-07-19 20:21:55.897737] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-xxx-client-6: disconnected from xxx-client-6. Client process will keep trying to connect to glusterd until brick's port is available
[2017-07-19 20:21:55.902838] I [MSGID: 108006] [afr-common.c:4131:afr_local_init] 0-xxx-replicate-5: no subvolumes up
[2017-07-19 20:21:55.940539] I [MSGID: 109063] [dht-layout.c:702:dht_layout_normalize] 0-xxx-dht: Found anomalies in / (gfid = 00000000-0000-0000-0000-000000000001). Holes=1 overlaps=0
[2017-07-19 20:21:55.940565] W [MSGID: 109005] [dht-selfheal.c:1782:dht_selfheal_directory] 0-xxx-dht: Directory selfheal failed: 1 subvolumes down.Not fixing. path = /, gfid = 00000000-0000-0000-0000-000000000001
[2017-07-19 20:21:55.940635] I [dht-rebalance.c:2651:gf_defrag_start_crawl] 0-xxx-dht: gf_defrag_start_crawl using commit hash 0
[2017-07-19 20:21:55.940674] I [MSGID: 108006] [afr-common.c:4131:afr_local_init] 0-xxx-replicate-5: no subvolumes up
[2017-07-19 20:21:56.021657] I [MSGID: 109081] [dht-common.c:3819:dht_setxattr] 0-xxx-dht: fixing the layout of /
[2017-07-19 20:21:56.021679] W [MSGID: 109016] [dht-selfheal.c:1474:dht_fix_layout_of_directory] 0-xxx-dht: Layout fix failed: 1 subvolume(s) are down. Skipping fix layout.
[2017-07-19 20:21:56.021906] E [MSGID: 109026] [dht-rebalance.c:2693:gf_defrag_start_crawl] 0-xxx-dht: fix layout on / failed
[2017-07-19 20:21:56.021930] I [MSGID: 109028] [dht-rebalance.c:3039:gf_defrag_status_get] 0-xxx-dht: Rebalance is failed. Time taken is 1.00 secs
[2017-07-19 20:21:56.021936] I [MSGID: 109028] [dht-rebalance.c:3043:gf_defrag_status_get] 0-xxx-dht: Files migrated: 0, size: 0, lookups: 0, failures: 1, skipped: 0
[2017-07-19 20:21:56.022077] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7dc5) [0x7face572adc5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x55ba9798c785] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x55ba9798c609] ) 0-: received signum (15), shutting down

Environment

glusterfs 3.*

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.