Termination condition in assoc array garbage collection can cause the system to crash

Solution Verified - Updated -

Issue

  • Seems our systems are rebooting due to a kernel bug
  • System is crashing with following oops:
:Version: 3.10.0-123.8.1.el7.x86_64
:BUG: unable to handle kernel NULL pointer dereference at 0000000000000018
:IP: [<ffffffff812cfbd7>] assoc_array_gc+0x2f7/0x540
:PGD 0 
:Oops: 0000 [#1] SMP 
:Modules linked in: fuse btrfs zlib_deflate raid6_pq xor vfat msdos fat xfs libcrc32c ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle tun bridge stp llc ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables nfsv3 nfs fscache snd_hda_codec_hdmi dm_mirror dm_region_hash dm_log dm_mod coretemp kvm_intel iTCO_wdt iTCO_vendor_support kvm snd_hda_codec_realtek crct10dif_pclmul crc32_pclmul snd_hda_codec_generic crc32c_intel ghash_clmulni_intel dcdbas snd_hda_intel aesni_intel lrw gf128mul nvidia(POF) glue_helper ablk_helper cryptd snd_hda_codec pcspkr serio_raw snd_hwdep sb_edac snd_seq edac_core snd_seq_device snd_pcm i2c_i801 snd_page_alloc snd_timer snd lpc_ich drm mfd_core mei_me
: soundcore mei i2c_core ntb shpchp usb_storage mperf nfsd auth_rpcgss nfs_acl lockd sunrpc uinput binfmt_misc ext4 mbcache jbd2 sd_mod sr_mod crc_t10dif cdrom crct10dif_common isci ahci libahci libsas e1000e scsi_transport_sas libata ptp pps_core
:CPU: 0 PID: 11518 Comm: kworker/0:0 Tainted: PF          O--------------   3.10.0-123.8.1.el7.x86_64 #1
:Hardware name: Dell Inc. Precision T3600/08HPGT, BIOS A07 11/08/2012
:Workqueue: events key_garbage_collector
:task: ffff88081f95e660 ti: ffff880036234000 task.ti: ffff880036234000
:RIP: 0010:[<ffffffff812cfbd7>]  [<ffffffff812cfbd7>] assoc_array_gc+0x2f7/0x540
:RSP: 0018:ffff880036235d40  EFLAGS: 00010206
:RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000108ca5
:RDX: ffff880669342060 RSI: 0000000000000001 RDI: ffff880036c57d80
:RBP: ffff880036235da8 R08: 0000000000000001 R09: 0000000000000003
:R10: ffffea002086d000 R11: ffffffff812cfb45 R12: 0000000000000000
:R13: 0000000000000000 R14: 0000000000000013 R15: 0000000000000001
:FS:  0000000000000000(0000) GS:ffff88082f200000(0000) knlGS:0000000000000000
:CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
:CR2: 0000000000000018 CR3: 00000000018d0000 CR4: 00000000000407f0
:DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
:DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
:Stack:
: 00000000000000fe 0000000000000003 ffff880036235db8 ffffffff81245b90
: ffff880815eb9c00 0000000000000000 ffff880036c57d83 0000000078cd5470
: ffff8808236a47a0 ffff8808236a4820 0000000054504b5e ffff880036235fd8
:Call Trace:
: [<ffffffff81245b90>] ? keyring_detect_cycle_iterator+0x30/0x30
: [<ffffffff81246f75>] keyring_gc+0x75/0x80
: [<ffffffff8124454f>] key_garbage_collector+0x17f/0x390
: [<ffffffff8107e03b>] process_one_work+0x17b/0x460
: [<ffffffff8107ee0b>] worker_thread+0x11b/0x400
: [<ffffffff8107ecf0>] ? rescuer_thread+0x400/0x400
: [<ffffffff81085aff>] kthread+0xcf/0xe0
: [<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
: [<ffffffff815f29ac>] ret_from_fork+0x7c/0xb0
: [<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
:Code: 08 4c 8b 22 0f 84 bf 00 00 00 41 83 c7 01 49 83 e4 fc 41 83 ff 0f 4c 89 65 c0 0f 8f 5a fe ff ff 48 8b 45 c0 4d 63 cf 49 83 c1 02 <4e> 8b 34 c8 4d 85 f6 0f 84 be 00 00 00 41 f6 c6 01 0f 84 92 00 
:RIP  [<ffffffff812cfbd7>] assoc_array_gc+0x2f7/0x540
: RSP <ffff880036235d40>

Environment

  • Red Hat Enterprise Linux 7.0 (RHEL)
  • kernel-3.10.0-123.8.1 to kernel-3.10.0-123.13.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content