A server frequently crashes in vol_ru_verification_data_unpack() due to severe stack overflows/corruption and slab corruption caused by Veritas VxVM (vxio) issue
Issue
- A server frequently crashes in vol_ru_verification_data_unpack() due to Veritas VxVM (vxio) issue.
[Fri May 16 23:47:53 UTC 2025] VxVM VVR vxio V-5-0-645 Failing reads on rvg aial03_rvg. Rlink rlk_aial03p_aial03_rvg is inconsistent. :
[Fri May 16 23:47:53 UTC 2025] VxVM VVR vxio V-5-0-645 Failing reads on rvg aial03_rvg. Rlink rlk_aial03p_aial03_rvg is inconsistent. :
[Fri May 16 23:47:53 UTC 2025] VxVM VVR vxio V-5-0-645 Failing reads on rvg aial03_rvg. Rlink rlk_aial03p_aial03_rvg is inconsistent. :
[Fri May 16 23:47:53 UTC 2025] VxVM VVR vxio V-5-0-645 Failing reads on rvg aial03_rvg. Rlink rlk_aial03p_aial03_rvg is inconsistent. :
[Fri May 16 23:47:53 UTC 2025] VxVM VVR vxio V-5-0-645 Failing reads on rvg aial03_rvg. Rlink rlk_aial03p_aial03_rvg is inconsistent. :
[Fri May 16 23:47:53 UTC 2025] VxVM VVR vxio V-5-0-645 Failing reads on rvg aial03_rvg. Rlink rlk_aial03p_aial03_rvg is inconsistent. :
[Fri May 16 23:55:05 UTC 2025] VxVM VVR vxio V-5-0-80 Cannot send out a message
[Sat May 17 00:26:03 UTC 2025] VxVM VVR vxio V-5-0-266 Rlink rlk_aial03p_aial03_rvg disconnected from remote :
[Sat May 17 00:26:07 UTC 2025] VxVM VVR vxio V-5-0-1406 Node 10.32.181.177 disconnected from node 10.214.180.120
[Sat May 17 00:26:10 UTC 2025] VxVM VVR vxio V-5-0-1402 Connected from node 10.32.181.177 to node 10.214.180.120
[Sat May 17 00:26:10 UTC 2025] VxVM VVR vxio V-5-0-265 Rlink rlk_aial03p_aial03_rvg connected to remote :
[Sat May 17 00:26:10 UTC 2025] VxVM VVR vxio V-5-0-1449 Disabling checksum for rlink rlk_aial03p_aial03_rvg
[Sat May 17 01:10:46 UTC 2025] BUG: unable to handle kernel paging request at ffff914a58287000
[Sat May 17 01:10:46 UTC 2025] PGD 9f802067 P4D 9f802067 PUD 9f803067 PMD 17631063 PTE 8000000018287061
[Sat May 17 01:10:46 UTC 2025] Oops: 0003 [#1] SMP PTI
[Sat May 17 01:10:46 UTC 2025] CPU: 3 PID: 5706 Comm: vxiod Kdump: loaded Tainted: P OE -------- - - 4.18.0-553.50.1.el8_10.x86_64 #1
[Sat May 17 01:10:46 UTC 2025] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
[Sat May 17 01:10:46 UTC 2025] RIP: 0010:vol_ru_verification_data_unpack+0x8e/0xe0 [vxio]
[Sat May 17 01:10:46 UTC 2025] Code: 0c 24 48 83 e0 f8 85 f6 7e 43 83 ee 01 48 8d 9a 90 00 00 00 48 8d ac b2 94 00 00 00 eb 14 8b 10 48 83 c3 04 48 83 c0 04 0f ca <89> 53 fc 48 39 dd 74 1b 48 39 c8 72 e7 48 89 e7 e8 0d dc ea ff 48
[Sat May 17 01:10:46 UTC 2025] RSP: 0018:ffffbbd1839f7d68 EFLAGS: 00010282
[Sat May 17 01:10:46 UTC 2025] RAX: ffff914c4f422fdc RBX: ffff914a58287004 RCX: ffff918c8f62c000
[Sat May 17 01:10:46 UTC 2025] RDX: 00000000d02f424f RSI: ffff914c4f240000 RDI: ffffbbd1839f7d68
[Sat May 17 01:10:46 UTC 2025] RBP: ffff914ad888c110 R08: ffffffffffff8000 R09: 0000000000000000
[Sat May 17 01:10:46 UTC 2025] R10: ffff914bbdd2d400 R11: 00002ef2efe92f80 R12: ffff914bbdd2d400
[Sat May 17 01:10:46 UTC 2025] R13: 0000000046203000 R14: ffff914c2a030000 R15: ffff914a58084000
[Sat May 17 01:10:46 UTC 2025] FS: 0000000000000000(0000) GS:ffff914e6fd80000(0000) knlGS:0000000000000000
[Sat May 17 01:10:46 UTC 2025] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Sat May 17 01:10:46 UTC 2025] CR2: ffff914a58287000 CR3: 000000009de10005 CR4: 00000000001706e0
[Sat May 17 01:10:46 UTC 2025] Call Trace:
[Sat May 17 01:10:46 UTC 2025] ? __die_body+0x1a/0x60
[Sat May 17 01:10:46 UTC 2025] ? no_context+0x1ba/0x3f0
[Sat May 17 01:10:46 UTC 2025] ? __bad_area_nosemaphore+0x157/0x180
[Sat May 17 01:10:46 UTC 2025] ? spurious_kernel_fault+0x1ed/0x250
[Sat May 17 01:10:46 UTC 2025] ? do_page_fault+0x37/0x12d
[Sat May 17 01:10:46 UTC 2025] ? page_fault+0x1e/0x30
[Sat May 17 01:10:46 UTC 2025] ? vol_ru_verification_data_unpack+0x8e/0xe0 [vxio]
[Sat May 17 01:10:46 UTC 2025] ? vol_ru_verification_data_unpack+0xa3/0xe0 [vxio]
[Sat May 17 01:10:46 UTC 2025] vol_ru_verify+0x9d/0x110 [vxio]
[Sat May 17 01:10:46 UTC 2025] volrv_seclog_bulk_cleanup_verification+0x107/0x1a0 [vxio]
[Sat May 17 01:10:46 UTC 2025] volrv_seclog_write1_done+0xbf/0xd0 [vxio]
[Sat May 17 01:10:46 UTC 2025] voliod_iohandle+0x21f/0x390 [vxio]
[Sat May 17 01:10:46 UTC 2025] voliod_loop+0xc2/0x340 [vxio]
[Sat May 17 01:10:46 UTC 2025] ? voliod_iohandle+0x390/0x390 [vxio]
[Sat May 17 01:10:46 UTC 2025] kthread+0x134/0x150
[Sat May 17 01:10:46 UTC 2025] ? set_kthread_struct+0x50/0x50
[Sat May 17 01:10:46 UTC 2025] ret_from_fork+0x35/0x40
[Sat May 17 01:10:46 UTC 2025] Modules linked in: ib_core bluetooth ecdh_generic rfkill mptcp_diag xsk_diag vsock_diag raw_diag af_packet_diag netlink_diag falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_17607(E) xfs nfsv3 nfs_acl nfs lockd grace fscache unix_diag nf_conntrack_netlink udp_diag nft_queue tcp_diag inet_diag nft_counter nft_log nft_ct nf_tables_set xt_LOG nf_log_syslog nf_conntrack_tftp nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 dmpaa(POE) vxspec(POE) vxio(POE) vxdmp(POE) vxodm(POE) amf(POE) vxcafs(POE) vxportal(POE) fdd(POE) vxfs(POE) storageapi(OE) veki(POE) nf_tables libcrc32c nfnetlink_queue nfnetlink_log nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vmw_vsock_vmci_transport vsock sunrpc intel_rapl_msr vmw_balloon intel_rapl_common sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel rapl joydev pcspkr vmw_vmci i2c_piix4 binfmt_misc ext4 mbcache jbd2 sr_mod cdrom ata_generic sd_mod t10_pi sg vmwgfx drm_ttm_helper ttm
[Sat May 17 01:10:46 UTC 2025] drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt serio_raw drm ata_piix libata vmxnet3 vmw_pvscsi dm_mod fuse
[Sat May 17 01:10:46 UTC 2025] Red Hat flags: eBPF/event eBPF/rawtrace
[Sat May 17 01:10:46 UTC 2025] CR2: ffff914a58287000
Environment
- Red Hat Enterprise Linux.
- Third-Party Module [vxio]
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.