RHEL8.7 system running kernel version 4.18.0-425.10.1.el8_7 hangs or kernel panic due to soft lockup while loading and initializing the ASMLib driver.
Issue
RHEL8.7 system running kernel version 4.18.0-425.10.1.el8_7
hangs or kernel panic due to soft lockup while loading and initializing the ASMLib driver
.
[ 138.743845] oracleasm: loading out-of-tree module taints kernel.
[ 171.997031] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:0:5]
[ 171.997065] Modules linked in: oracleasm(O) nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_tables_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink intel_rapl_msr intel_rapl_common kvm_intel kvm snd_hda_codec_generic ledtrig_audio virtio_gpu drm_shmem_helper snd_hda_intel irqbypass drm_kms_helper snd_intel_dspcfg snd_intel_sdw_acpi crct10dif_pclmul crc32_pclmul snd_hda_codec syscopyarea sysfillrect sysimgblt fb_sys_fops snd_hda_core drm ghash_clmulni_intel rapl snd_hwdep snd_pcm snd_timer snd joydev pcspkr virtio_balloon soundcore i2c_piix4 xfs libcrc32c ata_generic ata_piix libata crc32c_intel serio_raw virtio_scsi virtio_blk virtio_console virtio_net net_failover failover dm_mirror dm_region_hash dm_log dm_mod
[ 171.997299] CPU: 0 PID: 5 Comm: kworker/0:0 Kdump: loaded Tainted: G O --------- - - 4.18.0-425.10.1.el8_7.x86_64 #1
[ 171.997338] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014
[ 171.997370] Workqueue: events once_deferred
[ 171.997404] RIP: 0010:smp_call_function_single+0xd5/0xf0
[ 171.997426] Code: 00 00 00 75 3a c9 e9 7a be a5 00 48 89 74 24 10 48 89 e6 48 89 54 24 18 e8 a8 fe ff ff 8b 14 24 83 e2 01 74 0a f3 90 8b 14 24 <83> e2 01 75 f6 eb c4 8b 05 c6 84 2e 02 85 c0 75 84 0f 0b eb 80 e8
[ 171.997474] RSP: 0018:ffff9a25c0347d80 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
[ 171.997495] RAX: 0000000000000000 RBX: ffffffffb1a1c92a RCX: 0000000000000830
[ 171.997514] RDX: 0000000000000011 RSI: 00000000000000fb RDI: 0000000000000830
[ 171.997531] RBP: ffff9a25c0347dd8 R08: 0000000000000001 R09: 0000000000000001
[ 171.997550] R10: ffffffffb1a1c488 R11: ffffffffafec666c R12: 0000000000000000
[ 171.997575] R13: ffffffffb1a1c93a R14: 0000000000000000 R15: ffff8e149b73af08
[ 171.997595] FS: 0000000000000000(0000) GS:ffff8e14dfc00000(0000) knlGS:0000000000000000
[ 171.997615] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 171.997631] CR2: 000055881e6ba998 CR3: 0000000014c10005 CR4: 0000000000060ef0
[ 171.997653] Call Trace:
[ 171.997674] ? optimize_nops.isra.10+0xa0/0xa0
[ 171.997694] ? optimize_nops.isra.10+0xa0/0xa0
[ 171.997708] ? optimize_nops.isra.10+0xa0/0xa0
[ 171.997722] on_each_cpu+0x2b/0x60
[ 171.997736] text_poke_bp_batch+0x9f/0x190
[ 171.997756] text_poke_finish+0x1b/0x2a
[ 171.997770] arch_jump_label_transform_apply+0x16/0x30
[ 171.997790] static_key_disable_cpuslocked+0x58/0x90
[ 171.997814] static_key_disable+0x16/0x20
[ 171.997827] once_deferred+0x1a/0x30
[ 171.997845] process_one_work+0x1a7/0x360
[ 171.997866] worker_thread+0x30/0x390
[ 171.997878] ? create_worker+0x1a0/0x1a0
[ 171.997890] kthread+0x10b/0x130
[ 171.997910] ? set_kthread_struct+0x50/0x50
[ 171.997924] ret_from_fork+0x35/0x40
[ 171.998199] Kernel panic - not syncing: softlockup: hung tasks
[ 171.998430] CPU: 0 PID: 5 Comm: kworker/0:0 Kdump: loaded Tainted: G O L --------- - - 4.18.0-425.10.1.el8_7.x86_64 #1
[ 171.998942] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014
[ 171.999276] Workqueue: events once_deferred
[ 171.999644] Call Trace:
[ 172.000157] <IRQ>
[ 172.000498] dump_stack+0x41/0x60
[ 172.000827] panic+0xe7/0x2ac
[ 172.001067] ? __switch_to_asm+0x51/0x80
[ 172.001288] watchdog_timer_fn.cold.10+0x85/0x9e
[ 172.001517] ? watchdog+0x30/0x30
[ 172.001740] __hrtimer_run_queues+0x101/0x280
[ 172.001964] hrtimer_interrupt+0x100/0x220
[ 172.002166] smp_apic_timer_interrupt+0x6a/0x130
[ 172.002370] apic_timer_interrupt+0xf/0x20
[ 172.002563] </IRQ>
[ 172.002768] RIP: 0010:smp_call_function_single+0xd5/0xf0
[ 172.002959] Code: 00 00 00 75 3a c9 e9 7a be a5 00 48 89 74 24 10 48 89 e6 48 89 54 24 18 e8 a8 fe ff ff 8b 14 24 83 e2 01 74 0a f3 90 8b 14 24 <83> e2 01 75 f6 eb c4 8b 05 c6 84 2e 02 85 c0 75 84 0f 0b eb 80 e8
[ 172.003358] RSP: 0018:ffff9a25c0347d80 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
[ 172.003551] RAX: 0000000000000000 RBX: ffffffffb1a1c92a RCX: 0000000000000830
[ 172.003780] RDX: 0000000000000011 RSI: 00000000000000fb RDI: 0000000000000830
[ 172.003987] RBP: ffff9a25c0347dd8 R08: 0000000000000001 R09: 0000000000000001
[ 172.004180] R10: ffffffffb1a1c488 R11: ffffffffafec666c R12: 0000000000000000
[ 172.004369] R13: ffffffffb1a1c93a R14: 0000000000000000 R15: ffff8e149b73af08
[ 172.004563] ? __inet_hash_connect+0x9c/0x4d0
[ 172.004782] ? optimize_nops.isra.10+0xa0/0xa0
[ 172.004970] ? optimize_nops.isra.10+0xa0/0xa0
[ 172.005152] ? optimize_nops.isra.10+0xa0/0xa0
[ 172.005330] on_each_cpu+0x2b/0x60
[ 172.005506] text_poke_bp_batch+0x9f/0x190
[ 172.005696] text_poke_finish+0x1b/0x2a
[ 172.005873] arch_jump_label_transform_apply+0x16/0x30
[ 172.006054] static_key_disable_cpuslocked+0x58/0x90
[ 172.006236] static_key_disable+0x16/0x20
[ 172.006414] once_deferred+0x1a/0x30
[ 172.006609] process_one_work+0x1a7/0x360
[ 172.006791] worker_thread+0x30/0x390
[ 172.006969] ? create_worker+0x1a0/0x1a0
[ 172.007147] kthread+0x10b/0x130
[ 172.007325] ? set_kthread_struct+0x50/0x50
[ 172.007504] ret_from_fork+0x35/0x40
Environment
- Red Hat Enterprise Linux release 8.7, 8.6.z
- kernel-4.18.0-425.10.1.el8_7
- kernel-4.18.0-425.3.1.el8
- kmod-redhat-oracleasm-2.0.8-15.el8.x86_64
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.