RHGS - 'glusterd cannot start' or 'volumes cannot be deleted'

Solution In Progress - Updated -

Issue

When /var/lib/glusterd/vols is corrupted, then glusterd may fail to start, displaying the following messages:

[2016-05-05 15:48:38.557928] I [MSGID: 100030] [glusterfsd.c:1998:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.6.0.28 (args: /usr/sbin/glusterd --pid-file=/run/glusterd.pid)
[2016-05-05 15:48:38.564133] I [glusterd.c:1188:init] 0-management: Maximum allowed open file descriptors set to 65536
[2016-05-05 15:48:38.564183] I [glusterd.c:1233:init] 0-management: Using /var/lib/glusterd as working directory
[2016-05-05 15:48:38.568055] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device)
[2016-05-05 15:48:38.568086] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device
[2016-05-05 15:48:38.568098] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed
[2016-05-05 15:48:38.568264] W [rpcsvc.c:1524:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed
[2016-05-05 15:48:41.784623] I [glusterd-store.c:1944:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30000
[2016-05-05 15:48:42.777367] E [glusterd-store.c:2444:glusterd_store_update_volinfo] 0-management: Failed to get next store iter
[2016-05-05 15:48:42.777388] E [glusterd-store.c:2667:glusterd_store_retrieve_volume] 0-management: Failed to update volinfo for test-volume1 volume
[2016-05-05 15:48:42.777406] E [glusterd-store.c:2856:glusterd_store_retrieve_volumes] 0-: Unable to restore volume: test-volume1
[2016-05-05 15:48:42.777455] E [xlator.c:406:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again
[2016-05-05 15:48:42.777469] E [graph.c:322:glusterfs_graph_init] 0-management: initializing translator failed
[2016-05-05 15:48:42.777479] E [graph.c:525:glusterfs_graph_activate] 0-graph: init failed
[2016-05-05 15:48:42.778256] W [glusterfsd.c:1182:cleanup_and_exit] (--> 0-: received signum (0), shutting down

Also volumes may not be able to be deleted with the following message:

# gluster vol delete <vol>

Deleting volume will erase all information about the volume. Do you want to
continue? (y/n) y

volume delete: <vol>: failed: Staging failed on nodeb. Error:
Volume <vol> does not exist

Environment

Red Hat Gluster Storage Version 3.0, 3.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content