RHEL8.7 system running kernel version 4.18.0-425.10.1.el8_7 hangs or kernel panic due to soft lockup while loading and initializing the ASMLib driver.

Solution Verified - Updated -

Issue

RHEL8.7 system running kernel version 4.18.0-425.10.1.el8_7 hangs or kernel panic due to soft lockup while loading and initializing the ASMLib driver.

[  138.743845] oracleasm: loading out-of-tree module taints kernel.
[  171.997031] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:0:5]
[  171.997065] Modules linked in: oracleasm(O) nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_tables_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink intel_rapl_msr intel_rapl_common kvm_intel kvm snd_hda_codec_generic ledtrig_audio virtio_gpu drm_shmem_helper snd_hda_intel irqbypass drm_kms_helper snd_intel_dspcfg snd_intel_sdw_acpi crct10dif_pclmul crc32_pclmul snd_hda_codec syscopyarea sysfillrect sysimgblt fb_sys_fops snd_hda_core drm ghash_clmulni_intel rapl snd_hwdep snd_pcm snd_timer snd joydev pcspkr virtio_balloon soundcore i2c_piix4 xfs libcrc32c ata_generic ata_piix libata crc32c_intel serio_raw virtio_scsi virtio_blk virtio_console virtio_net net_failover failover dm_mirror dm_region_hash dm_log dm_mod
[  171.997299] CPU: 0 PID: 5 Comm: kworker/0:0 Kdump: loaded Tainted: G           O     --------- -  - 4.18.0-425.10.1.el8_7.x86_64 #1
[  171.997338] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014
[  171.997370] Workqueue: events once_deferred
[  171.997404] RIP: 0010:smp_call_function_single+0xd5/0xf0
[  171.997426] Code: 00 00 00 75 3a c9 e9 7a be a5 00 48 89 74 24 10 48 89 e6 48 89 54 24 18 e8 a8 fe ff ff 8b 14 24 83 e2 01 74 0a f3 90 8b 14 24 <83> e2 01 75 f6 eb c4 8b 05 c6 84 2e 02 85 c0 75 84 0f 0b eb 80 e8
[  171.997474] RSP: 0018:ffff9a25c0347d80 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
[  171.997495] RAX: 0000000000000000 RBX: ffffffffb1a1c92a RCX: 0000000000000830
[  171.997514] RDX: 0000000000000011 RSI: 00000000000000fb RDI: 0000000000000830
[  171.997531] RBP: ffff9a25c0347dd8 R08: 0000000000000001 R09: 0000000000000001
[  171.997550] R10: ffffffffb1a1c488 R11: ffffffffafec666c R12: 0000000000000000
[  171.997575] R13: ffffffffb1a1c93a R14: 0000000000000000 R15: ffff8e149b73af08
[  171.997595] FS:  0000000000000000(0000) GS:ffff8e14dfc00000(0000) knlGS:0000000000000000
[  171.997615] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  171.997631] CR2: 000055881e6ba998 CR3: 0000000014c10005 CR4: 0000000000060ef0
[  171.997653] Call Trace:
[  171.997674]  ? optimize_nops.isra.10+0xa0/0xa0
[  171.997694]  ? optimize_nops.isra.10+0xa0/0xa0
[  171.997708]  ? optimize_nops.isra.10+0xa0/0xa0
[  171.997722]  on_each_cpu+0x2b/0x60
[  171.997736]  text_poke_bp_batch+0x9f/0x190
[  171.997756]  text_poke_finish+0x1b/0x2a
[  171.997770]  arch_jump_label_transform_apply+0x16/0x30
[  171.997790]  static_key_disable_cpuslocked+0x58/0x90
[  171.997814]  static_key_disable+0x16/0x20
[  171.997827]  once_deferred+0x1a/0x30
[  171.997845]  process_one_work+0x1a7/0x360
[  171.997866]  worker_thread+0x30/0x390
[  171.997878]  ? create_worker+0x1a0/0x1a0
[  171.997890]  kthread+0x10b/0x130
[  171.997910]  ? set_kthread_struct+0x50/0x50
[  171.997924]  ret_from_fork+0x35/0x40
[  171.998199] Kernel panic - not syncing: softlockup: hung tasks
[  171.998430] CPU: 0 PID: 5 Comm: kworker/0:0 Kdump: loaded Tainted: G           O L   --------- -  - 4.18.0-425.10.1.el8_7.x86_64 #1
[  171.998942] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014
[  171.999276] Workqueue: events once_deferred
[  171.999644] Call Trace:
[  172.000157]  <IRQ>
[  172.000498]  dump_stack+0x41/0x60
[  172.000827]  panic+0xe7/0x2ac
[  172.001067]  ? __switch_to_asm+0x51/0x80
[  172.001288]  watchdog_timer_fn.cold.10+0x85/0x9e
[  172.001517]  ? watchdog+0x30/0x30
[  172.001740]  __hrtimer_run_queues+0x101/0x280
[  172.001964]  hrtimer_interrupt+0x100/0x220
[  172.002166]  smp_apic_timer_interrupt+0x6a/0x130
[  172.002370]  apic_timer_interrupt+0xf/0x20
[  172.002563]  </IRQ>
[  172.002768] RIP: 0010:smp_call_function_single+0xd5/0xf0
[  172.002959] Code: 00 00 00 75 3a c9 e9 7a be a5 00 48 89 74 24 10 48 89 e6 48 89 54 24 18 e8 a8 fe ff ff 8b 14 24 83 e2 01 74 0a f3 90 8b 14 24 <83> e2 01 75 f6 eb c4 8b 05 c6 84 2e 02 85 c0 75 84 0f 0b eb 80 e8
[  172.003358] RSP: 0018:ffff9a25c0347d80 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
[  172.003551] RAX: 0000000000000000 RBX: ffffffffb1a1c92a RCX: 0000000000000830
[  172.003780] RDX: 0000000000000011 RSI: 00000000000000fb RDI: 0000000000000830
[  172.003987] RBP: ffff9a25c0347dd8 R08: 0000000000000001 R09: 0000000000000001
[  172.004180] R10: ffffffffb1a1c488 R11: ffffffffafec666c R12: 0000000000000000
[  172.004369] R13: ffffffffb1a1c93a R14: 0000000000000000 R15: ffff8e149b73af08
[  172.004563]  ? __inet_hash_connect+0x9c/0x4d0
[  172.004782]  ? optimize_nops.isra.10+0xa0/0xa0
[  172.004970]  ? optimize_nops.isra.10+0xa0/0xa0
[  172.005152]  ? optimize_nops.isra.10+0xa0/0xa0
[  172.005330]  on_each_cpu+0x2b/0x60
[  172.005506]  text_poke_bp_batch+0x9f/0x190
[  172.005696]  text_poke_finish+0x1b/0x2a
[  172.005873]  arch_jump_label_transform_apply+0x16/0x30
[  172.006054]  static_key_disable_cpuslocked+0x58/0x90
[  172.006236]  static_key_disable+0x16/0x20
[  172.006414]  once_deferred+0x1a/0x30
[  172.006609]  process_one_work+0x1a7/0x360
[  172.006791]  worker_thread+0x30/0x390
[  172.006969]  ? create_worker+0x1a0/0x1a0
[  172.007147]  kthread+0x10b/0x130
[  172.007325]  ? set_kthread_struct+0x50/0x50
[  172.007504]  ret_from_fork+0x35/0x40

Environment

  • Red Hat Enterprise Linux release 8.7, 8.6.z
  • kernel-4.18.0-425.10.1.el8_7
  • kernel-4.18.0-425.3.1.el8
  • kmod-redhat-oracleasm-2.0.8-15.el8.x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content