Red Hat Enterprise Linux 6 で、大量の物理メモリが利用できてもメモリが足りなくなるためカーネルパニックが発生する
Issue
- メモリが不足したためカーネルパニックが発生し、以下のメッセージが表示されます。
db2adutl invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
db2adutl cpuset=/ mems_allowed=0
Pid:9295, comm: db2adutl Not tainted 2.6.32-358.14.1.el6.x86_64 #1
Call Trace:
[<ffffffff810cb561>] ? cpuset_print_task_mems_allowed+0x91/0xb0
[<ffffffff8111cd80>] ? dump_header+0x90/0x1b0
[<ffffffff8111ceee>] ? check_panic_on_oom+0x4e/0x80
[<ffffffff8111d5db>] ? out_of_memory+0x1bb/0x3c0
[<ffffffff8112c2ec>] ?__alloc_pages_nodemask+0x8ac/0x8d0
[<ffffffff811609ea>] ? alloc_pages_current+0xaa/0x110
[<ffffffff8111a167>] ?__page_cache_alloc+0x87/0x90
[<ffffffff8111ae94>] ? grab_cache_page_write_begin+0xa4/0xe0
[<ffffffffa0334277>] ? nfs_write_begin+0x77/0x220 [nfs]
[<ffffffffa0335575>] ? nfs_write_end+0x165/0x2b0 [nfs]
[<ffffffff8111a6e3>] ? generic_file_buffered_write+0x123/0x2e0
[<ffffffff8111c150>] ?__generic_file_aio_write+0x260/0x490
[<ffffffff81437923>] ? sock_recvmsg+0x133/0x160
[<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
[<ffffffff8111c408>] ? generic_file_aio_write+0x88/0x100
[<ffffffffa0334f9e>] ? nfs_file_write+0xde/0x1f0 [nfs]
[<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
[<ffffffff81180e1a>] ? do_sync_write+0xfa/0x140
[<ffffffff81096cc0>] ? autoremove_wake_function+0x0/0x40
[<ffffffff8121bce6>] ? security_file_permission+0x16/0x20
[<ffffffff81181118>] ? vfs_write+0xb8/0x1a0
[<ffffffff81181a11>] ? sys_write+0x51/0x90
[<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
CPU 8: hi: 0, btch: 1 usd: 0
CPU 9: hi: 0, btch: 1 usd: 0
CPU 10: hi: 0, btch: 1 usd: 0
CPU 11: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi:186, btch:31 usd: 1
CPU 1: hi:186, btch:31 usd:156
CPU 2: hi:186, btch:31 usd: 1
CPU 3: hi:186, btch:31 usd: 0
CPU 4: hi:186, btch:31 usd:166
CPU 5: hi:186, btch:31 usd: 0
CPU 6: hi:186, btch:31 usd: 0
CPU 7: hi:186, btch:31 usd:46
CPU 8: hi:186, btch:31 usd: 0
CPU 9: hi:186, btch:31 usd: 0
CPU 10: hi:186, btch:31 usd:43
CPU 11: hi:186, btch:31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi:186, btch:31 usd:30
CPU 1: hi:186, btch:31 usd:25
CPU 2: hi:186, btch:31 usd:30
CPU 3: hi:186, btch:31 usd: 0
CPU 4: hi:186, btch:31 usd:183
CPU 5: hi:186, btch:31 usd: 0
CPU 6: hi:186, btch:31 usd: 7
CPU 7: hi:186, btch:31 usd:51
CPU 8: hi:186, btch:31 usd: 0
CPU 9: hi:186, btch:31 usd: 0
CPU 10: hi:186, btch:31 usd:40
CPU 11: hi:186, btch:31 usd: 0
active_anon:176224 inactive_anon:626166 isolated_anon:0
active_file:501117 inactive_file:534914 isolated_file:0
unevictable:0 dirty:13089 writeback:521895 unstable:1512
free:26361 slab_reclaimable:6764 slab_unreclaimable:95522
mapped:603371 shmem:605375 pagetables:5346 bounce:0
Node 0 DMA free:15740kB min:124kB low:152kB high:184kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15276kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]:0 1913 7973 7973
Node 0 DMA32 free:40588kB min:16184kB low:20228kB high:24276kB active_anon:66664kB inactive_anon:110756kB active_file:669496kB inactive_file:685012kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1959300kB mlocked:0kB dirty:38244kB writeback:646920kB mapped:32884kB shmem:31404kB slab_reclaimable:6200kB slab_unreclaimable:108648kB kernel_stack:64kB pagetables:764kB unstable:5932kB bounce:0kB writeback_tmp:0kB pages_scanned:784160 all_unreclaimable? no
lowmem_reserve[]:0 0 6060 6060
Node 0 Normal free:49116kB min:51268kB low:64084kB high:76900kB active_anon:638232kB inactive_anon:2393908kB active_file:1334972kB inactive_file:1454644kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:6205440kB mlocked:0kB dirty:14112kB writeback:1440660kB mapped:2380600kB shmem:2390096kB slab_reclaimable:20856kB slab_unreclaimable:273440kB kernel_stack:4856kB pagetables:20620kB unstable:116kB bounce:0kB writeback_tmp:0kB pages_scanned:2113216 all_unreclaimable? no
lowmem_reserve[]:0 0 0 0
Node 0 DMA:3*4kB 2*8kB 2*16kB 2*32kB 2*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15740kB
Node 0 DMA32:505*4kB 7*8kB 1339*16kB 376*32kB 75*64kB 2*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 40588kB
Node 0 Normal:12279*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 49116kB
1641476 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 10485752kB
Total swap = 10485752kB
2097151 pages RAM
97810 pages reserved
1265671 pages shared
791217 pages non-shared
Environment
- Red Hat Enterprise Linux 6
- Kernel-2.6.32-358.14.1.el6
- OOM
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.