RHEL7: Kernel panic at cgroup_is_descendant+0x1d
Issue
- After frequent oom, kernel crashed with below logs:
[12581201.985136] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
[12581201.985655] [ 9720] 0 9720 44153 389 47 140 985 su
[12581201.985665] [ 9827] 0 9827 28811 343 15 67 985 bash
[12581201.985673] [ 9856] 0 9856 12611759 909872 5224 1074492 985 java
[12581201.985735] Memory cgroup out of memory: Kill process 12796 (java) score 1933 or sacrifice child
[12581201.987743] Killed process 9856 (java), UID 0, total-vm:50447036kB, anon-rss:3628704kB, file-rss:10912kB, shmem-rss:0kB
[12609882.427338] BUG: unable to handle kernel paging request at 0000001d00008040
[12609882.529072] IP: [<ffffffffb7126e1d>] cgroup_is_descendant+0x1d/0x40
[12609882.607703] PGD 0
[12609882.607803] Oops: 0000 [#1] SMP
[12609882.620181] Modules linked in: nfsv3 binfmt_misc tcp_diag inet_diag xt_set xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ip_set ipip tunnel4 ip_tunnel veth xt_statistic xt_nat xt_recent xt_comment xt_mark fuse nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables overlay(T) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache iptable_filter snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev snd_pcm snd_timer joydev snd sg virtio_rng soundcore pcspkr i2c_piix4 virtio_balloon parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace
[12609882.621025] sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common virtio_net net_failover virtio_console virtio_scsi failover ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci drm_panel_orientation_quirks virtio_ring serio_raw virtio floppy dm_mirror dm_region_hash dm_log dm_mod
[12609882.621363] CPU: 14 PID: 1 Comm: systemd Kdump: loaded Tainted: G ------------ T 3.10.0-1160.el7.x86_64 #1
[12609882.621448] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014
[12609882.621919] task: ffff8a8043b70000 ti: ffff8a804337c000 task.ti: ffff8a804337c000
[12609882.621989] RIP: 0010:[<ffffffffb7126e1d>] [<ffffffffb7126e1d>] cgroup_is_descendant+0x1d/0x40
[12609882.622073] RSP: 0018:ffff8a804337fcc0 EFLAGS: 00010286
[12609882.622135] RAX: 0000000000000000 RBX: ffffffffb7d5de80 RCX: 0000000000000020
[12609882.622202] RDX: ffff8a925f83c400 RSI: ffff8a803c791600 RDI: 0000001d00008000
[12609882.622269] RBP: ffff8a804337fcc0 R08: ffffffffb7d5de80 R09: 0000000000000010
[12609882.622344] R10: d5848d28d25d7b40 R11: 00000000000003ff R12: 00000000000181a0
[12609882.622432] R13: ffff8a9ebf2181a0 R14: 0000000000000010 R15: 0000000000000010
[12609882.622503] FS: 00007fde6652f940(0000) GS:ffff8a9ebf180000(0000) knlGS:0000000000000000
[12609882.622575] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12609882.622638] CR2: 00000000022a600e CR3: 000000015db54000 CR4: 00000000000006e0
[12609882.631524] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12609882.631603] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[12609882.631672] Call Trace:
[12609882.631764] [<ffffffffb723e55c>] __mem_cgroup_same_or_subtree+0x2c/0x40
[12609882.631839] [<ffffffffb723e60b>] drain_all_stock+0x9b/0x180
[12609882.639844] [<ffffffffb723e741>] mem_cgroup_reparent_charges+0x51/0x3c0
[12609882.639947] [<ffffffffb72668b8>] ? select_collect+0x68/0x120
[12609882.640014] [<ffffffffb72686a3>] ? d_walk+0xf3/0x310
[12609882.640077] [<ffffffffb723ec64>] mem_cgroup_css_offline+0x84/0x170
[12609882.640150] [<ffffffffb712ae8a>] cgroup_destroy_locked+0xea/0x370
[12609882.640219] [<ffffffffb712b132>] cgroup_rmdir+0x22/0x40
[12609882.640286] [<ffffffffb725b4ec>] vfs_rmdir+0xdc/0x150
[12609882.640349] [<ffffffffb7260171>] do_rmdir+0x1f1/0x220
[12609882.640433] [<ffffffffb7793ed5>] ? system_call_after_swapgs+0xa2/0x13a
[12609882.640501] [<ffffffffb7793ec9>] ? system_call_after_swapgs+0x96/0x13a
[12609882.640570] [<ffffffffb7793ed5>] ? system_call_after_swapgs+0xa2/0x13a
[12609882.640639] [<ffffffffb7793ec9>] ? system_call_after_swapgs+0x96/0x13a
[12609882.640709] [<ffffffffb7793ed5>] ? system_call_after_swapgs+0xa2/0x13a
[12609882.640785] [<ffffffffb7793ec9>] ? system_call_after_swapgs+0x96/0x13a
[12609882.640857] [<ffffffffb72613a6>] SyS_rmdir+0x16/0x20
[12609882.650724] [<ffffffffb7793f92>] system_call_fastpath+0x25/0x2a
[12609882.651794] [<ffffffffb7793ed5>] ? system_call_after_swapgs+0xa2/0x13a
[12609882.652842] Code: f7 ff 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 55 48 85 ff 48 89 e5 74 18 48 39 f7 75 0a eb 1b 0f 1f 00 48 39 fe 74 13 <48> 8b 7f 40 48 85 ff 75 f2 31 c0 5d c3 66 0f 1f 44 00 00 b8 01
[12609882.655163] RIP [<ffffffffb7126e1d>] cgroup_is_descendant+0x1d/0x40
[12609882.656235] RSP <ffff8a804337fcc0>
[12609882.657318] CR2: 0000001d00008040
Environment
- Red Hat Enterprise Linux 7
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.