glusterd fails to start due to no space
Issue
Glusterd does not come up on the gluster storage server node. It has following logs
[2016-12-15 11:00:12.786649] E [MSGID: 100017] [glusterfsd.c:1887:glusterfs_pidfile_setup] 0-glusterfsd: pidfile /var/run/glusterd.pid open failed [No space left on device]
[2016-12-15 11:00:38.711243] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.1 (args: /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid)
[2016-12-15 11:00:38.711969] E [MSGID: 100017] [glusterfsd.c:1887:glusterfs_pidfile_setup] 0-glusterfsd: pidfile /var/run/glusterd.pid open failed [No space left on device]
[2016-12-15 11:09:28.526025] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.1 (args: /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid)
[2016-12-15 11:09:28.526910] E [MSGID: 100017] [glusterfsd.c:1887:glusterfs_pidfile_setup] 0-glusterfsd: pidfile /var/run/glusterd.pid open failed [No space left on device]
[2016-12-15 11:09:53.554127] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.1 (args: /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid)
[2016-12-15 11:09:53.554883] E [MSGID: 100017] [glusterfsd.c:1887:glusterfs_pidfile_setup] 0-glusterfsd: pidfile /var/run/glusterd.pid open failed [No space left on device]
[2016-12-15 11:13:58.904807] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.1 (args: /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid)
[2016-12-15 11:13:58.917045] I [MSGID: 106478] [glusterd.c:1348:init] 0-management: Maximum allowed open file descriptors set to 65536
[2016-12-15 11:13:58.917098] I [MSGID: 106479] [glusterd.c:1397:init] 0-management: Using /var/lib/glusterd as working directory
[2016-12-15 11:13:58.925825] W [MSGID: 103071] [rdma.c:4590:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed [No such device]
[2016-12-15 11:13:58.925856] W [MSGID: 103055] [rdma.c:4896:init] 0-rdma.management: Failed to initialize IB Device
[2016-12-15 11:13:58.925867] W [rpc-transport.c:358:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed
[2016-12-15 11:13:58.925960] W [rpcsvc.c:1595:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed
[2016-12-15 11:13:58.925976] E [MSGID: 106243] [glusterd.c:1621:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2016-12-15 11:14:01.870498] E [MSGID: 101032] [store.c:435:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info. [No such file or directory]
[2016-12-15 11:14:01.870569] E [MSGID: 101032] [store.c:435:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info. [No such file or directory]
[2016-12-15 11:14:01.870575] I [MSGID: 106514] [glusterd-store.c:2058:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30703
[2016-12-15 11:14:01.918498] E [MSGID: 101032] [store.c:435:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/vols/RHS-ISO/bricks/ts2rhsn001-stor:-glustervol-ISO. [No such file or directory]
[2016-12-15 11:14:01.918532] E [MSGID: 106201] [glusterd-store.c:3021:glusterd_store_retrieve_volumes] 0-management: Unable to restore volume: RHS-ISO
[2016-12-15 11:14:01.918561] E [MSGID: 101019] [xlator.c:428:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again
[2016-12-15 11:14:01.918574] E [MSGID: 101066] [graph.c:326:glusterfs_graph_init] 0-management: initializing translator failed
[2016-12-15 11:14:01.918584] E [MSGID: 101176] [graph.c:672:glusterfs_graph_activate] 0-graph: init failed
[2016-12-15 11:14:01.919136] W [glusterfsd.c:1219:cleanup_and_exit] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xe5) [0x7f6d460e5ea5] -->/usr/sbin/glusterd(glusterfs_process_volfp+0x126) [0x7f6d460e5d66] -->/usr/sbin/glusterd(cleanup_and_exit+0x71) [0x7f6d460e5ba1] ) 0-: received signum (0), shutting down
Environment
- RHS-3.0
- RHGS-3.1
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
