Server reported crash in __ip_make_skb() using Red Hat Enterprise Linux Real Time Kernel
Issue
- Found below messages in "kernel ring buffer log" :
[ 2377.931578] BUG: unable to handle kernel NULL pointer dereference at 0000000000000020
[ 2377.931583] IP: [<ffffffff815afe4a>] __ip_make_skb+0x16a/0x420
[ 2377.931584] PGD 0
[ 2377.931586] Oops: 0000 [#1] PREEMPT SMP
[ 2377.931605] Modules linked in: binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxsf(OE) ppdev intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer pcspkr snd sg vboxguest(OE) soundcore i2c_piix4 i2c_core video parport_pc
[ 2377.931612] parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ahci libahci ata_piix e1000 libata fjes dm_mirror dm_region_hash dm_log dm_mod
[ 2377.931614] CPU: 1 PID: 4790 Comm: svn Tainted: G OE ------------ 3.10.0-514.16.1.rt56.437.el7.x86_64 #1
[ 2377.931615] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 2377.931615] task: ffff880058e59020 ti: ffff88004c050000 task.ti: ffff88004c050000
[ 2377.931617] RIP: 0010:[<ffffffff815afe4a>] [<ffffffff815afe4a>] __ip_make_skb+0x16a/0x420
[ 2377.931618] RSP: 0018:ffff88004c053b88 EFLAGS: 00010282
[ 2377.931619] RAX: 00000000ffffffff RBX: ffff880011d6fe00 RCX: 0000000000000040
[ 2377.931619] RDX: 0000000000000001 RSI: ffff88004c053c70 RDI: ffff8800662ee610
[ 2377.931620] RBP: ffff88004c053bc0 R08: 00000000000000c0 R09: ffffffff81acc740
[ 2377.931620] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88007a568a50
[ 2377.931621] R13: ffff88007a568580 R14: 0000000000000000 R15: 0000000000000028
[ 2377.931622] FS: 00007ff06c1ed8c0(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
[ 2377.931622] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2377.931623] CR2: 0000000000000020 CR3: 000000004c2d5000 CR4: 00000000000406e0
[ 2377.931626] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2377.931626] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2377.931627] Stack:
[ 2377.931628] 0000000000000000 ffff880000000040 ffff88007a568580 ffff88007a568580
[ 2377.931629] ffff88004c053d10 ffffffff81acc740 000000000f02000a ffff88004c053bd8
[ 2377.931630] ffffffff815b0170 ffff880011d6f100 ffff88004c053ce8 ffffffff815b0568
[ 2377.931630] Call Trace:
[ 2377.931633] [<ffffffff815b0170>] ip_push_pending_frames+0x20/0x40
[ 2377.931635] [<ffffffff815b0568>] ip_send_unicast_reply+0x268/0x2c0
[ 2377.931637] [<ffffffff81554f62>] ? skb_release_data+0xf2/0x140
[ 2377.931638] [<ffffffff815ce3f0>] ? tcp_v4_destroy_sock+0x250/0x2b0
[ 2377.931639] [<ffffffff815ce773>] tcp_v4_send_reset+0x323/0x470
[ 2377.931641] [<ffffffff815cea4e>] tcp_v4_do_rcv+0x18e/0x360
[ 2377.931643] [<ffffffff810b804b>] ? migrate_disable+0xab/0xf0
[ 2377.931646] [<ffffffff8154f729>] release_sock+0xa9/0x180
[ 2377.931647] [<ffffffff815bb56a>] tcp_close+0xaa/0x410
[ 2377.931649] [<ffffffff815e4e0b>] inet_release+0x6b/0x90
[ 2377.931650] [<ffffffff8154a85f>] sock_release+0x1f/0x80
[ 2377.931651] [<ffffffff8154a8d2>] sock_close+0x12/0x20
[ 2377.931653] [<ffffffff811f4988>] __fput+0xd8/0x250
[ 2377.931654] [<ffffffff811f4bee>] ____fput+0xe/0x10
[ 2377.931657] [<ffffffff810a2777>] task_work_run+0xa7/0xe0
[ 2377.931659] [<ffffffff8101ae2e>] do_notify_resume+0xae/0x100
[ 2377.931661] [<ffffffff81690401>] int_signal+0x12/0x17
[ 2377.931671] Code: 76 05 3c 05 0f 94 c2 0f b6 43 7c b9 40 00 00 00 83 e0 fe 09 d0 88 43 7c 41 0f b6 85 b7 04 00 00 83 e8 02 3c 01 76 42 44 8b 7b 68 <49> 8b 46 20 4c 89 f7 48 89 75 c8 4c 89 4d d0 ff 50 20 31 c9 41
[ 2377.931673] RIP [<ffffffff815afe4a>] __ip_make_skb+0x16a/0x420
[ 2377.931673] RSP <ffff88004c053b88>
[ 2377.931673] CR2: 0000000000000020
Environment
- Red Hat Enterprise Linux for Real Time.
- kernel-rt-3.10.0-514.16.1.rt56.437.el7.x86_64.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.