Issue

• Kernel panic at __kmalloc_track_caller+149.
• Logs captured before the panic:

[18262.046572] general protection fault: 0000 [#1] SMP 
[18262.046624] Modules linked in: nfsv3 nfs_acl nfs lockd sunrpc fscache bonding ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables ext4 mbcache jbd2 intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ipmi_devintf openafs(POF) crc32c_intel ghash_clmulni_intel ipmi_si i7core_edac aesni_intel lrw gf128mul glue_helper iTCO_wdt ablk_helper pcspkr cryptd iTCO_vendor_support ipmi_msghandler lpc_ich edac_core mfd_core i2c_i801 shpchp ioatdma acpi_cpufreq binfmt_misc xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_common cdrom usb_storage ast syscopyarea sysfillrect sysimgblt drm_kms_helper ttm ahci libahci qla2xxx mpt2sas igb drm libata ptp pps_core dca scsi_transport_fc raid_class i2c_algo_bit scsi_transport_sas scsi_tgt i2c_core dm_mirror dm_region_hash
[18262.047044]  dm_log dm_mod
[18262.047056] CPU: 10 PID: 15406 Comm: dsmc Tainted: PF       W IO--------------   3.10.0-229.1.2.el7.x86_64 #1
[18262.047094] Hardware name: Oracle Corporation SUN FIRE X4170 M2 SERVER        /ASSY,MOTHERBOARD,X4170, BIOS 08140105 01/28/2014
[18262.047136] task: ffff880c5b494440 ti: ffff880c594d0000 task.ti: ffff880c594d0000
[18262.047164] RIP: 0010:[<ffffffff811ae065>]  [<ffffffff811ae065>] __kmalloc_track_caller+0x95/0x230
[18262.047206] RSP: 0018:ffff880c594d3b50  EFLAGS: 00010282
[18262.047227] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 00000000000049c6
[18262.047253] RDX: 00000000000049c5 RSI: 0000000000000000 RDI: 0000000000000000
[18262.047280] RBP: ffff880c594d3b88 R08: 0000000000016380 R09: ffff880667803e00
[18262.047306] R10: ffff880c34b5a1f0 R11: 0000000000000000 R12: 00000000000000d0
[18262.047332] R13: 746e6f632f6d7264 R14: 0000000000000004 R15: ffff880667803e00
[18262.047359] FS:  00007f66797c4700(0000) GS:ffff880c7fc80000(0000) knlGS:0000000000000000
[18262.047389] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[18262.047411] CR2: 00007f66741eea20 CR3: 0000000c59742000 CR4: 00000000000007e0
[18262.047437] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18262.047464] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18262.047490] Stack:
[18262.047499]  ffff880667803e00 ffffffffa0a9b00c 0000000000000004 ffff880c34b5a1f0
[18262.047534]  0000000000000004 ffff880c0f62c088 ffff880c0f62c000 ffff880c594d3ba8
[18262.047568]  ffffffff81172ec0 ffff880c594d3d28 ffff880c594d3e68 ffff880c594d3cb0
[18262.047602] Call Trace:
[18262.047629]  [<ffffffffa0a9b00c>] ? nfs_readdir_page_filler+0x2bc/0x470 [nfs]
[18262.047663]  [<ffffffff81172ec0>] kmemdup+0x20/0x50
[18262.047686]  [<ffffffffa0a9b00c>] nfs_readdir_page_filler+0x2bc/0x470 [nfs]
[18262.047717]  [<ffffffffa0a9c417>] nfs_readdir_xdr_to_array+0x1f7/0x330 [nfs]
[18262.047747]  [<ffffffffa0a9c550>] ? nfs_readdir_xdr_to_array+0x330/0x330 [nfs]
[18262.047777]  [<ffffffffa0a9c572>] nfs_readdir_filler+0x22/0x90 [nfs]
[18262.048846]  [<ffffffff811571bf>] do_read_cache_page+0x7f/0x190
[18262.049871]  [<ffffffff811d9ec0>] ? fillonedir+0xe0/0xe0
[18262.050888]  [<ffffffff8115730c>] read_cache_page+0x1c/0x30
[18262.051908]  [<ffffffffa0a9c793>] nfs_readdir+0x1b3/0x650 [nfs]
[18262.052926]  [<ffffffffa0381a50>] ? nfs3_xdr_dec_getattr3res+0x80/0x80 [nfsv3]
[18262.053943]  [<ffffffff811d9ec0>] ? fillonedir+0xe0/0xe0
[18262.054949]  [<ffffffff811d9db0>] vfs_readdir+0xb0/0xe0
[18262.055947]  [<ffffffff811e61ce>] ? mntput_no_expire+0x3e/0x120
[18262.056957]  [<ffffffff811da1d5>] SyS_getdents+0x95/0x120
[18262.057943]  [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b
[18262.058918] Code: cc 00 00 49 8b 50 08 4d 8b 28 49 8b 40 10 4d 85 ed 0f 84 20 01 00 00 48 85 c0 0f 84 17 01 00 00 49 63 47 20 48 8d 4a 01 4d 8b 07 <49> 8b 5c 05 00 4c 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 b8 49 
[18262.061010] RIP  [<ffffffff811ae065>] __kmalloc_track_caller+0x95/0x230
[18262.062029]  RSP <ffff880c594d3b50>

• Another pattern is seen as below.

[187700.009183] BUG: unable to handle kernel paging request at ffff8803c22379a0
[187700.009196] IP: [<ffffffff811ae475>] __kmalloc_track_caller+0x95/0x230
[187700.009210] PGD 1ee5067 PUD 0 
[187700.009214] Oops: 0000 [#1] SMP 
[187700.009221] Modules linked in: arc4 md4 nls_utf8 cifs dns_resolver serio_raw dm_mirror dm_region_hash dm_log ext4 dm_mod mbcache jbd2 coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr shpchp i2c_piix4 vmw_vmci parport_pc parport nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_common vmwgfx drm_kms_helper ttm drm ahci vmxnet3 libahci ata_piix libata vmw_pvscsi i2c_core floppy
[187700.009280] CPU: 0 PID: 18841 Comm: db2fmpr Not tainted 3.10.0-229.20.1.el7.x86_64 #1
[187700.009284] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
[187700.009287] task: ffff8801ba730000 ti: ffff8802f8d8c000 task.ti: ffff8802f8d8c000
[187700.009289] RIP: 0010:[<ffffffff811ae475>]  [<ffffffff811ae475>] __kmalloc_track_caller+0x95/0x230
[187700.009295] RSP: 0018:ffff8802f8d8fdb8  EFLAGS: 00010286
[187700.009297] RAX: 0000000000000000 RBX: 0000000000000018 RCX: 0000000000c30dff
[187700.009299] RDX: 0000000000c30dfe RSI: 0000000000000000 RDI: 0000000000000002
[187700.009301] RBP: ffff8802f8d8fdf0 R08: 00000000000163c0 R09: ffff880333001c00
[187700.009303] R10: ffff8801c0ef5100 R11: 0000000000000000 R12: 00000000000000d0
[187700.009305] R13: ffff8803c22379a0 R14: 0000000000000018 R15: ffff880333001c00
[187700.009308] FS:  00007f29e30d3740(0000) GS:ffff88033fc00000(0000) knlGS:0000000000000000
[187700.009310] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[187700.009312] CR2: ffff8803c22379a0 CR3: 000000032de9c000 CR4: 00000000000007f0
[187700.009361] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[187700.009387] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[187700.009388] Stack:
[187700.009390]  ffff880333001c00 ffffffff8126de5b 0000000000000018 ffff8801c2237220
[187700.009395]  ffff8801c0ef5100 0000000000000000 ffff8801ba730000 ffff8802f8d8fe10
[187700.009399]  ffffffff811732a0 ffff88023538dcc0 ffff8801ba730000 ffff8802f8d8fe28
[187700.009403] Call Trace:
[187700.009411]  [<ffffffff8126de5b>] ? selinux_cred_prepare+0x1b/0x30
[187700.009418]  [<ffffffff811732a0>] kmemdup+0x20/0x50
[187700.009422]  [<ffffffff8126de5b>] selinux_cred_prepare+0x1b/0x30
[187700.009429]  [<ffffffff81269a66>] security_prepare_creds+0x16/0x20
[187700.009436]  [<ffffffff8109e0e6>] prepare_creds+0xf6/0x1c0
[187700.009439]  [<ffffffff8109e68f>] prepare_exec_creds+0xf/0x40
[187700.009446]  [<ffffffff811ce159>] prepare_bprm_creds+0x39/0x80
[187700.009450]  [<ffffffff811ce29c>] do_execve_common.isra.22+0xfc/0x690
[187700.009454]  [<ffffffff811d6ddf>] ? getname_flags+0x4f/0x1a0
[187700.009457]  [<ffffffff811d6e00>] ? getname_flags+0x70/0x1a0
[187700.009462]  [<ffffffff811ceac9>] SyS_execve+0x29/0x30
[187700.009469]  [<ffffffff816149a9>] stub_execve+0x69/0xa0
[187700.009471] Code: cc 00 00 49 8b 50 08 4d 8b 28 49 8b 40 10 4d 85 ed 0f 84 20 01 00 00 48 85 c0 0f 84 17 01 00 00 49 63 47 20 48 8d 4a 01 4d 8b 07 <49> 8b 5c 05 00 4c 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 b8 49 
[187700.009514] RIP  [<ffffffff811ae475>] __kmalloc_track_caller+0x95/0x230
[187700.009518]  RSP <ffff8802f8d8fdb8>
[187700.009520] CR2: ffff8803c22379a0

An additional pattern observed:

[1886505.631957] general protection fault: 0000 [#1] SMP 
[1886505.631980] Modules linked in: bridge stp llc team_mode_activebackup team intel_powerclamp coretemp xfs libcrc32c iTCO_wdt iTCO_vendor_support vfat fat intel_rapl kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ipmi_devintf ablk_helper cryptd pcspkr sb_edac edac_core lpc_ich i2c_i801 hpwdt mfd_core hpilo ioatdma dca shpchp wmi ipmi_si ipmi_msghandler pcc_cpufreq acpi_power_meter binfmt_misc sg ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm drm tg3 ptp i2c_core pps_core hpsa dm_mirror dm_region_hash dm_log dm_mod
[1886505.632193] CPU: 5 PID: 33100 Comm: splunkd Not tainted 3.10.0-229.el7.x86_64 #1
[1886505.632215] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 12/27/2015
[1886505.632238] task: ffff880c0e36db00 ti: ffff880581b64000 task.ti: ffff880581b64000
[1886505.632259] RIP: 0010:[<ffffffff811ac485>]  [<ffffffff811ac485>] __kmalloc+0x95/0x230
[1886505.632286] RSP: 0018:ffff880581b67a30  EFLAGS: 00010286
[1886505.632302] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00000000bfa3842d
[1886505.632323] RDX: 00000000bfa3842c RSI: 0000000000000000 RDI: 0000000000000d17
[1886505.632343] RBP: ffff880581b67a60 R08: 00000000000164e0 R09: ffffffffa07bb977
[1886505.632363] R10: ffff880c4f403300 R11: ffffffffa07bba55 R12: 0000000000008250
[1886505.632382] R13: 7c04000000000000 R14: 0000000000000d18 R15: ffff880c4f403300
[1886505.632403] FS:  00007facea5ff700(0000) GS:ffff880c4faa0000(0000) knlGS:0000000000000000
[1886505.632426] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1886505.632442] CR2: 00007face0dd0f60 CR3: 000000170d20c000 CR4: 00000000001407e0
[1886505.632463] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1886505.632484] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1886505.632503] Stack:
[1886505.632511]  ffffffffa07bb977 0000000000000000 0000000000008250 0000000000000d18
[1886505.632539]  0000000000000000 ffff880c2126b308 ffff880581b67a90 ffffffffa07bb977
[1886505.633347]  ffff8817060ed768 0000000000000000 ffff88044e00cf80 ffff880c250edd60
[1886505.634119] Call Trace:
[1886505.634887]  [<ffffffffa07bb977>] ? kmem_alloc+0x77/0xf0 [xfs]
[1886505.635628]  [<ffffffffa07bb977>] kmem_alloc+0x77/0xf0 [xfs]
[1886505.636374]  [<ffffffffa08021ae>] xfs_log_commit_cil+0x39e/0x4c0 [xfs]
[1886505.637126]  [<ffffffffa07bb1bc>] xfs_trans_commit+0x11c/0x240 [xfs]
[1886505.637846]  [<ffffffffa07ef677>] xfs_create+0x637/0x680 [xfs]
[1886505.638556]  [<ffffffffa07b0a7b>] xfs_vn_mknod+0xbb/0x1e0 [xfs]
[1886505.639291]  [<ffffffffa07b0bd3>] xfs_vn_create+0x13/0x20 [xfs]
[1886505.639984]  [<ffffffff811d28ad>] vfs_create+0xcd/0x130
[1886505.640698]  [<ffffffff811d5e7f>] do_last+0xb8f/0x1270
[1886505.641377]  [<ffffffff810b1c4c>] ? update_curr+0xcc/0x150
[1886505.642078]  [<ffffffff811d6622>] path_openat+0xc2/0x4c0
[1886505.642744]  [<ffffffff81012609>] ? __switch_to+0x179/0x4a0
[1886505.643402]  [<ffffffff811d7a5b>] do_filp_open+0x4b/0xb0
[1886505.644056]  [<ffffffff811e44a7>] ? __alloc_fd+0xa7/0x130
[1886505.644719]  [<ffffffff811c58b3>] do_sys_open+0xf3/0x1f0
[1886505.645376]  [<ffffffff811c59ce>] SyS_open+0x1e/0x20
[1886505.646077]  [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b
[1886505.646764] Code: cc 00 00 49 8b 50 08 4d 8b 28 49 8b 40 10 4d 85 ed 0f 84 30 01 00 00 48 85 c0 0f 84 27 01 00 00 49 63 42 20 48 8d 4a 01 4d 8b 02 <49> 8b 5c 05 00 4c 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 b8 49 
[1886505.648260] RIP  [<ffffffff811ac485>] __kmalloc+0x95/0x230
[1886505.648959]  RSP <ffff880581b67a30>