Kernel panic at exception RIP kmem_cache_free+546 and kfree+276

Issue

• Kernel panic at BUG at mm/slab.c:535! with below traces:

[421508.918570] kernel BUG at mm/slab.c:535!
[421508.923495] invalid opcode: 0000 [#1] SMP 
[421508.928637] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:06:00.1/net/eth1/statistics/tx_window_errors
[421508.941278] CPU 3 
[421508.943425] Modules linked in: rdma_ucm(U) ib_ucm(U) rdma_cm(U) iw_cm(U) ib_ipoib(U) ib_cm(U) ib_uverbs(U) ib_umad(U) mlx5_ib(U) mlx5_core(U) mlx4_en(U) mlx4_ib(U) ib_sa(U) ib_mad(U) ib_core(U) ib_addr(U) mlx4_core(U) compat(U) knem(U) nls_utf8 8021q garp stp llc nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ipmi_devintf ipmi_si ipmi_msghandler iTCO_wdt iTCO_vendor_support microcode ics932s401 cdc_ether usbnet mii sg lpc_ich mfd_core shpchp igb dca i2c_algo_bit i2c_core ptp pps_core ext4 jbd2 mbcache sd_mod crc_t10dif wmi megaraid_sas dm_mirror dm_region_hash dm_log dm_mod [last unloaded: compat]
[421509.008252] 
[421509.010458] Pid: 17458, comm: sbd-java Not tainted 2.6.32-504.3.3.el6.x86_64 #1 IBM IBM System x3650 M4: -[7915AC1]-/00Y8473
[421509.024018] RIP: 0010:[<ffffffff811754d2>]  [<ffffffff811754d2>] kmem_cache_free+0x222/0x2b0
[421509.034442] RSP: 0018:ffff88082bdbda98  EFLAGS: 00010046
[421509.040917] RAX: ffffea001c696af8 RBX: ffff8808745cac00 RCX: ffff88080c0d8c00
[421509.049957] RDX: 0000000000000000 RSI: ffff88081e1e93c0 RDI: ffff88081e1e93c0
[421509.058916] RBP: ffff88082bdbdaf8 R08: 0000000000000000 R09: 0000000000000000
[421509.067871] R10: ffff8807d1a3aa40 R11: 0000000000000130 R12: 0000000000000282
[421509.076823] R13: ffff88081e1e93c0 R14: ffff8812744f0540 R15: ffff8807d1a3ae9c
[421509.085780] FS:  00007f2d1456b700(0000) GS:ffff8800282c0000(0000) knlGS:0000000000000000
[421509.095802] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[421509.102764] CR2: 00007f2f57e93038 CR3: 00000012738a1000 CR4: 00000000001407e0
[421509.111719] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[421509.120753] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[421509.129720] Process sbd-java (pid: 17458, threadinfo ffff88082bdbc000, task ffff8808102a0080)
[421509.140228] Stack:
[421509.143014]  0000000000000130 ffff88080c0d8c00 ffff88082bdbdb18 0000000000000001
[421509.151666] <d> ffff88081e1e93c0 ffff88081e1e93c0 ffff8807d1a3aaf0 ffff88081e1e93c0
[421509.161268] <d> 0000000000000000 0000000000000000 ffff8807d1a3aaf0 ffff8807d1a3ae9c
[421509.171532] Call Trace:
[421509.174806]  [<ffffffff8144ff97>] __kfree_skb+0x47/0xa0
[421509.181176]  [<ffffffff814a7dd3>] tcp_recvmsg+0xf63/0x1060
[421509.187844]  [<ffffffff8152997e>] ? thread_return+0x4e/0x7d0
[421509.194709]  [<ffffffff814c8c3a>] inet_recvmsg+0x5a/0x90
[421509.201184]  [<ffffffff8144a083>] sock_recvmsg+0x133/0x160
[421509.207855]  [<ffffffff8106d504>] ? enqueue_task_fair+0x64/0x100
[421509.215111]  [<ffffffff810586cd>] ? check_preempt_curr+0x6d/0x90
[421509.222364]  [<ffffffff8109eb00>] ? autoremove_wake_function+0x0/0x40
[421509.230103]  [<ffffffff81064bc0>] ? wake_up_state+0x10/0x20
[421509.236861]  [<ffffffff810b235c>] ? wake_futex+0x3c/0x60
[421509.243342]  [<ffffffff8144a1fe>] sys_recvfrom+0xee/0x180
[421509.249912]  [<ffffffff810b56cb>] ? sys_futex+0x7b/0x170
[421509.256390]  [<ffffffff8100c675>] ? math_state_restore+0x45/0x60
[421509.263640]  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[421509.270897] Code: 79 3b 00 41 8b 07 41 3b 47 04 8b 55 b0 74 4f 48 8b 4d c8 89 c2 83 c0 01 49 89 4c d7 18 41 89 07 66 ff 03 66 66 90 e9 98 fe ff ff <0f> 0b eb fe 48 8b 40 10 48 8b 10 e9 55 fe ff ff 4c 8d 7b 18 89 
[421509.294383] RIP  [<ffffffff811754d2>] kmem_cache_free+0x222/0x2b0
[421509.301751]  RSP <ffff88082bdbda98>