[RHEL8.1/8.2] The kernel crashes due to slab corruption frequently with Symantec SEP modules
Issue
- The kernel crashes frequently where Symantec Endpoint Protection (SEP) for Linux is running.
- Panic message 1
[ 106.623133] ------------[ cut here ]------------
[ 106.623138] kernel BUG at mm/slub.c:3910!
[ 106.625375] invalid opcode: 0000 [#1] SMP NOPTI
[ 106.627294] CPU: 0 PID: 2031 Comm: rtvscand Kdump: loaded Tainted: P W OE --------- - - 4.18.0-193.6.3.el8_
2.x86_64 #1
[ 106.631530] Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1501130705 01/13/2015
[ 106.635684] RIP: 0010:kfree+0x148/0x160
[ 106.637354] Code: 89 d7 5d 41 5c e9 88 72 fd ff 48 89 d9 48 89 da 41 b8 01 00 00 00 5b 4c 89 d6 5d 41 5c e9 b0 f6 ff ff 49 8b 42 08 a8 01 75 c5 <0f> 0b 48 8b 3d 8f bf f7 00 e9 e7 fe ff ff 66 2e 0f 1f 84 00 00 00
[ 106.640868] RSP: 0018:ffffb31782323e50 EFLAGS: 00210246
[ 106.642487] RAX: fffff19184dc5008 RBX: ffff9545763d8778 RCX: 00000000000006d6
[ 106.644132] RDX: 0000000000000000 RSI: ffff9545770a2f00 RDI: 00006abb40000000
[ 106.645768] RBP: ffffffff8fe84700 R08: 00003dd20423b650 R09: ffffffff8f2ed851
[ 106.647378] R10: fffff19184d8f600 R11: 0000000000000001 R12: ffffffff8f366a4d
[ 106.649207] R13: ffff9545763d8778 R14: 0000000000000000 R15: ffff954571ac8a40
[ 106.651152] FS: 0000000000000000(0000) GS:ffff95457ac00000(0063) knlGS:00000000e74d2b40
[ 106.653004] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
[ 106.654616] CR2: 000056022b122320 CR3: 0000000118084000 CR4: 00000000000006f0
[ 106.656336] Call Trace:
[ 106.658045] full_proxy_release+0x6d/0xb0
[ 106.660119] __fput+0xb7/0x230
[ 106.661715] task_work_run+0x8a/0xb0
[ 106.663409] exit_to_usermode_loop+0xeb/0xf0
[ 106.664892] do_fast_syscall_32+0x29a/0x2e1
[ 106.666532] entry_SYSCALL_compat_after_hwframe+0x45/0x4d
[ 106.668285] Modules linked in: symap_custom_4_18_0_193_6_3_el8_2_x86_64(POE) symev_custom_4_18_0_193_6_3_el8_2_x86_64(OE) vmw_vsock_vmci_transport vsock vfat fat ext4 mbcache jbd2 ppdev vmw_balloon edac_mce_amd joydev pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c vmwgfx drm_kms_helper sr_mod cdrom syscopyarea ata_generic sysfillrect sysimgblt fb_sys_fops ttm sd_mod sg drm ata_piix serio_raw libata vmxnet3 vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
- Panic message 2
[ 194.633116] general protection fault: 0000 [#1] SMP NOPTI
[ 194.635272] CPU: 1 PID: 2036 Comm: rtvscand Kdump: loaded Tainted: P OE --------- - - 4.18.0-193.6.3.el8_2.x86_64 #1
[ 194.638581] Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1501130705 01/13/2015
[ 194.642416] RIP: 0010:__x86_indirect_thunk_rax+0x10/0x20
[ 194.644203] Code: 60 2f c8 ff b9 f2 ff ff ff 30 c0 e9 65 31 c8 ff b9 f2 ff ff ff e9 67 31 c8 ff e8 07 00 00 00 f3 90 0f ae e8 eb f9 48 89 04 24 <c3> 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 e8 07 00 00 00 f3
[ 194.647370] RSP: 0018:ffffa66e40eeff00 EFLAGS: 00210282
[ 194.648867] RAX: f6d7d4e1a5cd5229 RBX: ffff93b31e779e00 RCX: ffff93b31e710358
[ 194.650717] RDX: ffff93b31e779e00 RSI: ffff93b31e7102c0 RDI: ffff93b31e779e00
[ 194.652381] RBP: ffff93b31e7102c0 R08: 00000000f7ef6000 R09: 0000000000000001
[ 194.654122] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
[ 194.655732] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 194.657453] FS: 0000000000000000(0000) GS:ffff93b33ac40000(0063) knlGS:00000000e744fb40
[ 194.659292] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
[ 194.661067] CR2: 00007f5ae4aa0000 CR3: 000000011e772000 CR4: 00000000000006e0
[ 194.662796] Call Trace:
[ 194.664320] ? filp_close+0x31/0x60
[ 194.665869] ? __ia32_sys_close+0x1e/0x50
[ 194.667556] ? do_fast_syscall_32+0xa7/0x2e1
[ 194.669144] ? entry_SYSCALL_compat_after_hwframe+0x45/0x4d
[ 194.670840] Modules linked in: symap_custom_4_18_0_193_6_3_el8_2_x86_64(POE) symev_custom_4_18_0_193_6_3_el8_2_x86_64(OE) vmw_vsock_vmci_transport vsock vfat fat ext4 mbcache jbd2 ppdev vmw_balloon edac_mce_amd joydev pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c vmwgfx sr_mod drm_kms_helper cdrom ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops sd_mod ttm sg drm ata_piix libata serio_raw vmxnet3 vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
- Panic message 3
[80470.323111] stack segment: 0000 [#1] SMP NOPTI
[80470.323115] CPU: 4 PID: 457267 Comm: kworker/u12:2 Kdump: loaded Tainted: P OE --------- - - 4.18.0-193.14.3.el8_2.x86_64 #1
[80470.323116] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[80470.323125] Workqueue: writeback wb_workfn (flush-253:0)
[80470.323132] RIP: 0010:__kmalloc+0xa0/0x200
[80470.323133] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 5f a6 77 5a 49 8b 28 48 85 ed 0f 84 21 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b
[80470.323134] RSP: 0018:ffffad0d8352b6b0 EFLAGS: 00010206
[80470.323135] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000002028
[80470.323136] RDX: 0000000000002027 RSI: 0000000000600640 RDI: ffff931547c02400
[80470.323137] RBP: 0100000087c20d00 R08: 000000000002e1c0 R09: 0000000000000000
[80470.323138] R10: ffff9315f798e278 R11: 0000000000000001 R12: 0000000000600640
[80470.323139] R13: 0000000000001080 R14: ffff931547c02400 R15: ffff931547c02400
[80470.323140] FS: 0000000000000000(0000) GS:ffff9315f9b00000(0000) knlGS:0000000000000000
[80470.323141] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[80470.323142] CR2: 00007ff0f80ab000 CR3: 000000004860a004 CR4: 00000000001606e0
[80470.323192] Call Trace:
[80470.323282] ? kmem_alloc+0x64/0x150 [xfs]
[80470.323313] ? kmem_alloc_large+0x29/0x90 [xfs]
[80470.323341] kmem_alloc+0x64/0x150 [xfs]
[80470.323379] ? xfs_log_commit_cil+0x12d/0x630 [xfs]
[80470.323406] kmem_alloc_large+0x29/0x90 [xfs]
[80470.323434] xfs_log_commit_cil+0x12d/0x630 [xfs]
[80470.323466] ? xfs_bmapi_convert_delalloc+0x333/0x460 [xfs]
[80470.323494] __xfs_trans_commit+0xa4/0x350 [xfs]
[80470.323515] xfs_bmapi_convert_delalloc+0x333/0x460 [xfs]
[80470.323551] xfs_map_blocks+0x168/0x410 [xfs]
[80470.323580] xfs_do_writepage+0x121/0x470 [xfs]
[80470.323587] write_cache_pages+0x1aa/0x440
[80470.323612] ? xfs_vm_releasepage+0x80/0x80 [xfs]
[80470.323617] ? submit_bio+0x45/0x140
[80470.323641] ? xfs_setfilesize_trans_alloc.isra.17+0x3d/0x90 [xfs]
[80470.323666] xfs_vm_writepages+0x64/0xa0 [xfs]
[80470.323669] do_writepages+0x41/0xd0
[80470.323672] __writeback_single_inode+0x3d/0x360
[80470.323674] writeback_sb_inodes+0x1e3/0x450
[80470.323676] __writeback_inodes_wb+0x5d/0xb0
[80470.323678] wb_writeback+0x25f/0x2f0
[80470.323682] ? cpumask_next+0x17/0x20
[80470.323683] wb_workfn+0x342/0x400
[80470.323686] process_one_work+0x1a7/0x3b0
[80470.323688] worker_thread+0x30/0x390
[80470.323689] ? create_worker+0x1a0/0x1a0
[80470.323691] kthread+0x112/0x130
[80470.323693] ? kthread_flush_work_fn+0x10/0x10
[80470.323696] ret_from_fork+0x1f/0x40
[80470.323698] Modules linked in: binfmt_misc nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace symap_custom_4_18_0_193_14_3_el8_2_x86_64(POE) fscache nf_nat_ftp nft_objref nf_conntrack_ftp nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_tables_set nft_chain_nat_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nft_chain_route_ipv6 nft_chain_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack nft_chain_route_ipv4 ip6_tables nft_compat ip_set nf_tables nfnetlink symev_custom_4_18_0_193_14_3_el8_2_x86_64(OE) vmw_vsock_vmci_transport vsock sunrpc intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel vmw_balloon joydev intel_rapl_perf pcspkr vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod sg ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci crc32c_intel serio_raw ata_piix vmxnet3 vmw_pvscsi
[80470.323736] libata dm_mirror dm_region_hash dm_log dm_mod
Environment
- Red Hat Enterprise Linux 8.1
- kernel-4.18.0-147.el8
-
Symantec Endpoint Protection (SEP) for Linux
- symap_custom_4_18_0_147_el8_x86_64
- symev_custom_4_18_0_147_el8_x86_64
-
Red Hat Enterprise Linux 8.2
- kernel-4.18.0-193.6.3.el8_2
-
Symantec Endpoint Protection (SEP) for Linux
- symev_custom_4_18_0_193_6_3_el8_2_x86_64
- symap_custom_4_18_0_193_6_3_el8_2_x86_64)
-
Red Hat Enterprise Linux 8.2
- kernel-4.18.0-193.14.3.el8_2
- Symantec Endpoint Protection (SEP) for Linux
- symev_custom_4_18_0_193_14_3_el8_2_x86_64
- symap_custom_4_18_0_193_14_3_el8_2_x86_64
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.