Kernel crashed due to list corruption coming out of the stall_tbl_insert() function provided by "cb_appc_events_4_1_2640849".

Solution Unverified - Updated -

Issue

  • The server rebooted unexpectedly.
  • The server crashed in a third-party module cb_appc_events_4_1_2640849 code path.
  • Kernel crashed with the following logs:

crash> log [773032.574534] list_add corruption. next->prev should be prev (ffff8f503008f1e8), but was 0000000000000000. (next=ffff8f50c7763e18). [773032.574565] ------------[ cut here ]------------ [773032.574568] kernel BUG at lib/list_debug.c:25! [773032.574580] invalid opcode: 0000 [#1] SMP PTI [773032.574589] CPU: 2 PID: 2878618 Comm: Thread-121 Kdump: loaded Tainted: G W OE -------- - - 4.18.0-553.53.1.el8_10.x86_64 #1 [773032.574597] Hardware name: HP ProLiant BL460c Gen9, BIOS I36 07/18/2022 [773032.574602] RIP: 0010:__list_add_valid.cold.0+0x12/0x28 [773032.574616] Code: 00 48 8b 50 08 48 39 f2 0f 85 46 00 00 00 b8 01 00 00 00 c3 cc cc cc cc 48 89 d1 48 c7 c7 a0 50 74 99 48 89 c2 e8 32 41 c7 ff <0f> 0b 48 89 c1 4c 89 c6 48 c7 c7 f8 50 74 99 e8 1e 41 c7 ff 0f 0b [773032.574623] RSP: 0018:ffffb7d50dbc3b48 EFLAGS: 00010046 [773032.574632] RAX: 0000000000000075 RBX: ffff8f5b773c7600 RCX: 0000000000000000 [773032.574638] RDX: 0000000000000000 RSI: ffff8f5f3fc9e698 RDI: ffff8f5f3fc9e698 [773032.574644] RBP: ffff8f50adad0f80 R08: 0000000000000000 R09: c0000000ffff7fff [773032.574649] R10: 0000000000000001 R11: ffffb7d50dbc3968 R12: ffff8f5b773c7618 [773032.574654] R13: ffff8f5356962400 R14: 000000000000f1e0 R15: ffff8f503008f1e0 [773032.574659] FS: 00007f5eaa104700(0000) GS:ffff8f5f3fc80000(0000) knlGS:0000000000000000 [773032.574666] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [773032.574670] CR2: 00007fc8d5f41000 CR3: 0000000390060005 CR4: 00000000003706e0 [773032.574677] Call Trace: [773032.574685] ? __die_body+0x1a/0x60 [773032.574698] ? die+0x2a/0x50 [773032.574706] ? do_trap+0xe7/0x110 [773032.574714] ? __list_add_valid.cold.0+0x12/0x28 [773032.574722] ? do_invalid_op+0x36/0x40 [773032.574729] ? __list_add_valid.cold.0+0x12/0x28 [773032.574735] ? invalid_op+0x14/0x20 [773032.574750] ? __list_add_valid.cold.0+0x12/0x28 [773032.574754] ? __list_add_valid.cold.0+0x12/0x28 [773032.574760] stall_tbl_insert+0x20c/0x2c0 [cb_appc_events_4_1_2640849] [773032.574779] dynsec_wait_event_timeout+0x78/0x4b0 [cb_appc_events_4_1_2640849] [773032.574793] ? prepare_dynsec_event+0x1ad/0x1f0 [cb_appc_events_4_1_2640849] [773032.574807] dynsec_file_open+0x2a7/0x4b0 [cb_appc_events_4_1_2640849] [773032.574820] security_file_open+0x26/0xf0 [773032.574828] do_dentry_open+0xde/0x3a0 [773032.574837] path_openat+0x55b/0x1580 [773032.574847] do_filp_open+0x93/0x100 [773032.574855] ? getname_flags+0x4a/0x1e0 [773032.574860] ? __check_object_size+0xac/0x173 [773032.574866] ? path_get+0x11/0x30 [773032.574872] ? __alloc_fd+0x44/0x150 [773032.574880] do_sys_openat2+0x211/0x2b0 [773032.574888] do_sys_open+0x4b/0x80 [773032.574896] do_syscall_64+0x5b/0x1a0 [773032.574904] entry_SYSCALL_64_after_hwframe+0x66/0xcb [773032.574912] RIP: 0033:0x7f5ffb9dcf46 [773032.574921] Code: 89 54 24 08 e8 9b f4 ff ff 8b 74 24 0c 48 8b 3c 24 41 [773032.574928] RSP: 002b:00007f5eaa1015e0 EFLAGS: 00000293 ORIG_RAX: 0000000000000101 [773032.574935] RAX: ffffffffffffffda RBX: 0000000000000241 RCX: 00007f5ffb9dcf46 [773032.574939] RDX: 0000000000000241 RSI: 00007f5eaa101740 RDI: 00000000ffffff9c [773032.574942] RBP: 00007f5eaa101740 R08: 0000000000000000 R09: 0000000000000000 [773032.574945] R10: 00000000000001b6 R11: 0000000000000293 R12: 0000000004300d98 [773032.574948] R13: 00000000000001b6 R14: 00007f5ffbdc6420 R15: 0000000000000000 [773032.574953] Modules linked in: udp_diag tcp_diag inet_diag cb_appc_events_4_1_2640849(OE) binfmt_misc xt_CHECKSUM ipt_MASQUERADE xt_conntrack ipt_REJECT nf_reject_ipv4 nft_compat nft_counter nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables nfnetlink bridge stp llc bonding tls cfg80211 rfkill ext4 mbcache jbd2 dm_service_time dm_multipath intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel iTCO_wdt iTCO_vendor_support kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel rapl intel_cstate intel_uncore pcspkr lpc_ich i2c_i801 ipmi_ssif hpwdt ioatdma dca ses enclosure hpilo acpi_ipmi wmi ipmi_si ipmi_devintf acpi_power_meter ipmi_msghandler acpi_tad auth_rpcgss sunrpc xfs libcrc32c lpfc mgag200 drm_kms_helper syscopyarea sd_mod sysfillrect sysimgblt sg i2c_algo_bit drm_shmem_helper nvmet_fc drm nvmet nvme_fc crc32c_intel nvme_fabrics serio_raw nvme_core hpsa [773032.575073] be2net t10_pi scsi_transport_fc scsi_transport_sas dm_mod fuse [773032.575086] Red Hat flags: eBPF/event

Environment

  • Red Hat Enterprise Linux 8.10.
  • cb_appc_events_4_1_2640849 kernel module.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content