ストレージデバイスを多数持つシステムで Out of Memory エラーが発生したため、kdump カーネルがカーネルパニックを発生させる

Issue

• ストレージデバイスを多数持つシステムで Out of Memory エラーが発生したため、kdump カーネルがカーネルパニックを発生させます。

  以下は、ログのサンプルとなります。

  [   35.923346] systemd-journal invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
  [   35.933233] systemd-journal cpuset=/ mems_allowed=0
  [   35.939321] CPU:0 PID:329 Comm: systemd-journal Tainted:GF          O--------------   3.10.0-229.el7.x86_64 #1
  [   35.965850]  ffff88002d3b0000 000000003c774d89 ffff88002fcbf960 ffffffff81603f36
  [   35.974856]  ffff88002fcbf9f0 ffffffff815feeff ffff88002fc77bb0 ffff88002fc77bc8
  [   35.983850]  ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803
  [   35.992838] Call Trace:
  [   35.996214]  [<ffffffff81603f36>] dump_stack+0x19/0x1b
  [   36.002593]  [<ffffffff815feeff>] dump_header+0x8e/0x214
  [   36.009160]  [<ffffffff81117803>] ? delayacct_end+0x73/0xb0
  [   36.016015]  [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0
  [   36.023158]  [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30
  [   36.030886]  [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0
  [   36.037739]  [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90
  [   36.045466]  [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140
  [   36.052416]  [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0
  [   36.059359]  [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120
  [   36.066877]  [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90
  [   36.074586]  [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0
  [   36.081807]  [<ffffffff81171a6a>] shme3]  [<ffffffff811c8d18>] ?__sb_start_write+0x58/0x110
  [   36.096056]  [<ffffffff811c4c17>] do_fallocate+0x117/0x1b0
  [   36.102766]  [<ffffffff811c4d0b>] SyS_fallocate+0x5b/0xa0
  [   36.109368]  [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b
  [   36.116640] Mem-Info:
  [   36.119726] Node 0 DMA per-cpu:
  [   36.123810] CPU    0: hi:    0, btch:   1 usd:   0
  [   36.129712] Node 0 DMA32 per-cpu:
  [   36.133985] CPU    0: hi:   42, btch:   7 usd:13
  [   36.139887] active_anon:1164 inactive_anon:2319 isolated_anon:0
  [   36.139887]  active_file:0 inactive_file:0 isolated_file:0
  [   36.139887]  unevictable:10887 dirty:0 writeback:0 unstable:0
  [   36.139887]  free:501 slab_reclaimable:1764 slab_unreclaimable:10079
  [   36.139887]  mapped:913 shmem:2336 pagetables:154 bounce:0
  [   36.139887]  free_cma:0
  [   36.178206] Node 0 DMA free:516kB min:4kB low:4kB high:4kB active_anon:0kB inale:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:600kB managed:516kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimab   4450 le:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
  [   36.223329] lowmem_reserve[]:0 140 140 140
  [   36.228671] Node 0 DMA32 free:1488kB min:1508kB low:1884kB high:2260kB active_anon:4656kB inactive_anon:9276kB active_file:0kB inactive_file:0kB unevictable:43548kB isolated(anon):0kB isolated(file):0kB present:165268kB managed:144972kB mlocked:0kB dirty:0kB wr   4452 iteback:0kB mapped:3652kB shmem:9344kB slab_reclaimable:7056kB slab_unreclaimable:40316kB kernel_stack:2096kB pagetables:616kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
  [   36.278393] lowmem_reserve[]:0 0 0 0
  [   36.283193] Node 0 DMA:1*4kB (U) 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 516kB
  [   36.297908] Node 0 DMA32:30*4kB (UE) 23*8kB (UEM) 8*16kB (UE) 9*32kB (UEM) 8*64kB (U) 0*128kB 1*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1488kB
  [   36.314547] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
  [   36.324653] 13223 total pagecache pages
  [   36.329605] 0 pages in swap cache
  [   36.333974] Swap cache stats: add 0, delete 0, find 0/0
  [   36.340482] Free swap  = 0kB
  [   36.344360] Total swap = 0kB
  [   36.348234] 41467 pages RAM
  [   36.352003] 0 pages HighMem/MovableOnly
  [   36.356936] 5095 pages reserved
  [   36.361089] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
  [   36.370499] [  178]     0   178     7335      123      17        0             0 iscsid
  [   36.380107] [  219]     0   219     9179      881      22       220]     0   220     8732      325      20        0             0 systemd-udevd
  [   36.400681] [  221]     0   221     8732      308      20        0             0 systemd-udevd
  [   36.410964] [  228]     0   228     3018      446      15        0             0 dracut-initqueu
  [   36.421437] [  234]     0   234     8178      286      18        0             0 udevadm
  [   36.431133] [  329]     0   329     7156      317      18        0             0 systemd-journal
  [   36.441603] Out of memory:Kill process 228 (dracut-initqueu) score 12 or sacrifice child
  [   36.451402] Killed process 234 (udevadm) total-vm:32712kB, anon-rss:272kB, file-rss:872kB
  :
  [   47.018753] Kernel panic - not syncing:Out of memory and no killable processes...
  [   47.018753] 
  [   47.03md-udevd Tainted:GF          O--------------   3.10.0-229.el7.x86_64 #1
  [   47.056576]  ffffffff81837648 00000000275977a3 ffff88002febba50 ffffffff81603f36
  [   47.065632]  ffff88002febbad0 ffffffff815fd7da ffff880000000008 ffff88002febbae0
  [   47.074676]  ffff88002febba80 00000000275977a3 000000000000023a 0000000000000046
  [   47.083724] Call Trace:
  [   47.087164]  [<ffffffff81603f36>] dump_stack+0x19/0x1b
  [   47.093627]  [<ffffffff815fd7da>] panic+0xd8/0x1e7
  [   47.099696]  [<ffffffff8115a8da>] out_of_memory+0x4ea/0x4f0
  [   47.106645]  [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90
  [   47.114467]  [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140
  [   47.121519]  [<ffffffff8117ffa5>] do_wp_page+0xd5/0x7e0
  [   47.128078]  [<ffffffff81182734>] handle_mm_fault+0x584/0xd60
  [   47.135218]  [<ffffffff8160f1e6>] __do_page_fault+0x156/0x520
  [   47.142354]  [<ffffffff8160f5ca>] ? do_page_fault+0x1a/0x70
  [   47.149290]  [<ffffffff8160b808>] ? page_fault+0x28/0x30
  [   47.155925]  [<ffffffff810ad726>] ?__dequeue_entity+0x26/0x40
  [   47.163143]  [<ffffffff81012609>] ?__switch_to+0x179/0x4a0
  [   47.170047]  [<ffffffff8160f5ca>] do_page_fault+0x1a/0x70
  [   47.176745]  [<ffffffff8160b808>] page_fault+0x28/0x30
  [   47.311817] drm_kms_helper: panic occurred, switching back to text console
  [   47.342750] Rebooting in 10 seconds..