RHGS volume gets unmounted while copying large data to the volume
Issue
- The gluster volume is getting unmounted automatically
-
The
/var/log/messageshowing glusterfs getting killed by OOM killerJul 21 13:00:18 rhgs-clnt kernel: lowmem_reserve[]: 0 0 5050 5050 Jul 21 13:00:18 rhgs-clnt kernel: Node 0 Normal free:42260kB min:42316kB low:52892kB high:63472kB active_anon:4298296kB inactive_anon:716488kB active_file:348kB inactive_file:960kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:5171200kB mlocked:0kB dirty:0kB writeback:0kB mapped:452kB shmem:36kB slab_reclaimable:11016kB slab_unreclaimable:66248kB kernel_stack:3200kB pagetables:28620kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Jul 21 13:00:18 rhgs-clnt kernel: lowmem_reserve[]: 0 0 0 0 Jul 21 13:00:18 rhgs-clnt kernel: Node 0 DMA: 1*4kB 1*8kB 0*16kB 1*32kB 2*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15660kB Jul 21 13:00:18 rhgs-clnt kernel: Node 0 DMA32: 703*4kB 486*8kB 382*16kB 311*32kB 181*64kB 61*128kB 13*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 45484kB Jul 21 13:00:18 rhgs-clnt kernel: Node 0 Normal: 10558*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 42232kB Jul 21 13:00:18 rhgs-clnt kernel: 76956 total pagecache pages Jul 21 13:00:18 rhgs-clnt kernel: 76572 pages in swap cache Jul 21 13:00:18 rhgs-clnt kernel: Swap cache stats: add 11613481, delete 11536909, find 19061073/20276356 Jul 21 13:00:18 rhgs-clnt kernel: Free swap = 0kB Jul 21 13:00:18 rhgs-clnt kernel: Total swap = 1048572kB Jul 21 13:00:18 rhgs-clnt kernel: 2097136 pages RAM Jul 21 13:00:18 rhgs-clnt kernel: 81838 pages reserved Jul 21 13:00:18 rhgs-clnt kernel: 3251 pages shared Jul 21 13:00:18 rhgs-clnt kernel: 1984758 pages non-shared Jul 21 13:00:18 rhgs-clnt kernel: [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 21 13:00:18 rhgs-clnt kernel: [ 452] 0 452 2809 1 1 -17 -1000 udevd Jul 21 13:00:18 rhgs-clnt kernel: [ 1145] 0 1145 24346 41 1 -17 -1000 auditd Jul 21 13:00:18 rhgs-clnt kernel: [ 1147] 0 1147 20509 52 0 0 0 audispd Jul 21 13:00:18 rhgs-clnt kernel: [ 1168] 0 1168 113433 1 1 0 0 nslcd Jul 21 13:00:18 rhgs-clnt kernel: [ 1184] 0 1184 64047 788 0 0 0 rsyslogd Jul 21 13:00:18 rhgs-clnt kernel: [ 1219] 0 1219 4575 45 1 0 0 irqbalance Jul 21 13:00:18 rhgs-clnt kernel: [ 1237] 32 1237 4744 17 0 0 0 rpcbind Jul 21 13:00:18 rhgs-clnt kernel: [ 1259] 29 1259 5837 2 0 0 0 rpc.statd Jul 21 13:00:18 rhgs-clnt kernel: [ 1286] 81 1286 8472 2 1 0 0 dbus-daemon Jul 21 13:00:18 rhgs-clnt kernel: [ 1409] 0 1409 1020 1 1 0 0 acpid Jul 21 13:00:18 rhgs-clnt kernel: [ 1421] 68 1421 10533 113 1 0 0 hald Jul 21 13:00:18 rhgs-clnt kernel: [ 1422] 0 1422 5100 2 0 0 0 hald-runner Jul 21 13:00:18 rhgs-clnt kernel: [ 1454] 0 1454 5630 2 1 0 0 hald-addon-inpu Jul 21 13:00:18 rhgs-clnt kernel: [ 1464] 68 1464 4502 2 1 0 0 hald-addon-acpi Jul 21 13:00:18 rhgs-clnt kernel: [ 1479] 0 1479 53189 45 1 0 0 sssd Jul 21 13:00:18 rhgs-clnt kernel: [ 1480] 0 1480 62413 293 1 0 0 sssd_be Jul 21 13:00:18 rhgs-clnt kernel: [ 1481] 0 1481 54575 71 0 0 0 sssd_nss Jul 21 13:00:18 rhgs-clnt kernel: [ 1482] 0 1482 50939 75 1 0 0 sssd_pam Jul 21 13:00:18 rhgs-clnt kernel: [ 1483] 0 1483 52256 60 1 0 0 sssd_ssh Jul 21 13:00:18 rhgs-clnt kernel: [ 1484] 0 1484 50393 33 0 0 0 sssd_sudo Jul 21 13:00:18 rhgs-clnt kernel: [ 1530] 0 1530 26827 1 1 0 0 rpc.rquotad Jul 21 13:00:18 rhgs-clnt kernel: [ 1535] 0 1535 5417 1 1 0 0 rpc.mountd Jul 21 13:00:18 rhgs-clnt kernel: [ 1580] 0 1580 6292 2 0 0 0 rpc.idmapd Jul 21 13:00:18 rhgs-clnt kernel: [ 1869] 0 1869 15964 86 0 0 0 vmtoolsd Jul 21 13:00:18 rhgs-clnt kernel: [ 1909] 0 1909 16558 25 1 -17 -1000 sshd Jul 21 13:00:18 rhgs-clnt kernel: [ 1920] 0 1920 5429 18 0 0 0 xinetd Jul 21 13:00:18 rhgs-clnt kernel: [ 1944] 38 1944 8205 39 0 0 0 ntpd Jul 21 13:00:18 rhgs-clnt kernel: [ 1956] 0 1956 42102 39 1 0 0 vnetd Jul 21 13:00:18 rhgs-clnt kernel: [ 1960] 0 1960 51762 43 1 0 0 bpcd Jul 21 13:00:18 rhgs-clnt kernel: [ 1991] 5076 1991 10363 12 0 0 0 nrpe Jul 21 13:00:18 rhgs-clnt kernel: [ 2012] 0 2012 22737 63 0 0 0 sendmail Jul 21 13:00:18 rhgs-clnt kernel: [ 2022] 51 2022 20081 31 0 0 0 sendmail Jul 21 13:00:18 rhgs-clnt kernel: [ 2050] 0 2050 45255 2 0 0 0 abrtd Jul 21 13:00:18 rhgs-clnt kernel: [ 2060] 0 2060 44631 22 1 0 0 abrt-dump-oops Jul 21 13:00:18 rhgs-clnt kernel: [ 2071] 0 2071 49714 15 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [ 2091] 0 2091 51398 294 0 0 0 osad Jul 21 13:00:18 rhgs-clnt kernel: [ 2103] 0 2103 29216 28 0 0 0 crond Jul 21 13:00:18 rhgs-clnt kernel: [ 2117] 0 2117 25232 27 0 0 0 rhnsd Jul 21 13:00:18 rhgs-clnt kernel: [ 2128] 41 2128 54506 2 1 0 0 mailmanctl Jul 21 13:00:18 rhgs-clnt kernel: [ 2129] 41 2129 83976 15539 1 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2130] 41 2130 98478 12895 0 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2131] 41 2131 59263 1630 0 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2132] 41 2132 76469 14538 0 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2133] 41 2133 54565 615 0 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2134] 41 2134 78888 11628 1 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2135] 41 2135 80235 11255 0 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2136] 41 2136 54544 115 1 0 0 python Jul 21 13:00:18 rhgs-clnt kernel: [ 2148] 0 2148 6285 2 1 0 0 oddjobd Jul 21 13:00:18 rhgs-clnt kernel: [ 2163] 0 2163 1016 2 1 0 0 mingetty Jul 21 13:00:18 rhgs-clnt kernel: [ 2165] 0 2165 1016 2 1 0 0 mingetty Jul 21 13:00:18 rhgs-clnt kernel: [ 2167] 0 2167 1016 2 1 0 0 mingetty Jul 21 13:00:18 rhgs-clnt kernel: [ 2169] 0 2169 1016 2 0 0 0 mingetty Jul 21 13:00:18 rhgs-clnt kernel: [ 2171] 0 2171 3105 1 0 -17 -1000 udevd Jul 21 13:00:18 rhgs-clnt kernel: [ 2173] 0 2173 1016 2 1 0 0 mingetty Jul 21 13:00:18 rhgs-clnt kernel: [ 2175] 0 2175 1016 2 1 0 0 mingetty Jul 21 13:00:18 rhgs-clnt kernel: [ 2334] 0 2334 32831 2 0 0 0 screen Jul 21 13:00:18 rhgs-clnt kernel: [ 2335] 0 2335 27110 2 1 0 0 bash Jul 21 13:00:18 rhgs-clnt kernel: [ 7905] 0 7905 2808 1 0 -17 -1000 udevd Jul 21 13:00:18 rhgs-clnt kernel: [26396] 48 26396 50366 283 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26397] 48 26397 50308 262 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26398] 48 26398 50367 282 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26399] 48 26399 50303 266 0 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26400] 48 26400 50308 284 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26401] 48 26401 50366 264 0 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26402] 48 26402 50409 278 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [26403] 48 26403 50641 282 0 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [24104] 48 24104 50335 266 0 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [24121] 48 24121 50341 256 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [24268] 48 24268 50308 263 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [25299] 48 25299 50573 283 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [28054] 48 28054 50308 276 0 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [28055] 48 28055 50641 281 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [28056] 48 28056 50308 290 1 0 0 httpd Jul 21 13:00:18 rhgs-clnt kernel: [16587] 0 16587 2042236 1731318 0 0 0 glusterfs Jul 21 13:00:18 rhgs-clnt kernel: [10820] 0 10820 49787 809 1 0 0 bpbkar Jul 21 13:00:18 rhgs-clnt kernel: [ 6667] 41 6667 38651 108 1 0 0 crond Jul 21 13:00:18 rhgs-clnt kernel: [ 6668] 0 6668 38651 108 1 0 0 crond Jul 21 13:00:18 rhgs-clnt kernel: [ 6675] 0 6675 26524 35 1 0 0 sh Jul 21 13:00:18 rhgs-clnt kernel: [ 6677] 41 6677 73232 21481 0 0 0 gate_news Jul 21 13:00:18 rhgs-clnt kernel: [ 6678] 0 6678 90281 38494 0 0 0 genaliases Jul 21 13:00:18 rhgs-clnt kernel: [ 6681] 0 6681 27567 87 0 0 0 check_mk_agent Jul 21 13:00:18 rhgs-clnt kernel: Out of memory: Kill process 16587 (glusterfs) score 798 or sacrifice child Jul 21 13:00:18 rhgs-clnt kernel: Killed process 16587, UID 0, (glusterfs) total-vm:8168944kB, anon-rss:6925160kB, file-rss:124kB
Environment
Red Hat Enterprise Linux 7.2
Red Hat Gluster Storage 3.1.2
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Close
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.