why does our GlusterNFS server go down periodically?

Solution Unverified - Updated -

Issue

  • We periodically seem to see a situation where the gluster NFS server seems to die, and the only way i know to get it started it started again is to restart the glusterd (rolling) on the 4 RHS servers
Jul 29 00:25:35 rhs-01 kernel: glusterfs invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 29 00:25:35 rhs-01 kernel: glusterfs cpuset=/ mems_allowed=0-1
Jul 29 00:25:35 rhs-01 kernel: Pid: 20172, comm: glusterfs Not tainted 2.6.32-358.41.1.el6.x86_64 #1
Jul 29 00:25:35 rhs-01 kernel: Call Trace:
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff810cb681>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8111cef0>] ? dump_header+0x90/0x1b0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff810e93ce>] ? __delayacct_freepages_end+0x2e/0x30
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8121d46c>] ? security_real_capable_noaudit+0x3c/0x70
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8111d372>] ? oom_kill_process+0x82/0x2a0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8111d2b1>] ? select_bad_process+0xe1/0x120
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8111d7b0>] ? out_of_memory+0x220/0x3c0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8112c49f>] ? __alloc_pages_nodemask+0x89f/0x8d0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff81160e3a>] ? alloc_pages_vma+0x9a/0x150
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff81143fdb>] ? handle_pte_fault+0x76b/0xb50
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8114824f>] ? vma_adjust+0x11f/0x5e0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff811445fa>] ? handle_mm_fault+0x23a/0x310
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff810474e9>] ? __do_page_fault+0x139/0x480
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff8114ab8a>] ? do_mmap_pgoff+0x33a/0x380
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff815144be>] ? do_page_fault+0x3e/0xa0
Jul 29 00:25:35 rhs-01 kernel: [<ffffffff81511875>] ? page_fault+0x25/0x30
Jul 29 00:25:35 rhs-01 kernel: Mem-Info:
Jul 29 00:25:35 rhs-01 kernel: Node 0 DMA per-cpu:

Environment

Red Hat Storage Server - 2.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content