System crash due to hang with blocking RPC timer code on CPU

watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [kworker/u34:0:214718]
....
CPU: 15 PID: 214718 Comm: kworker/u34:0 Kdump: loaded Tainted: P        W  OEL   --------- -  - 4.18.0-240.15.1.el8_3.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: rpciod rpc_async_schedule [sunrpc]
RIP: 0010:mod_delayed_work_on+0x5f/0x90
....
Call Trace:
 ? rpc_wake_up_task_on_wq_queue_action_locked+0x270/0x270 [sunrpc]
 __rpc_add_timer+0x74/0xa0 [sunrpc]
 ? rpc_wake_up_next_func+0x10/0x10 [sunrpc]
 ? rpc_wake_up_task_on_wq_queue_action_locked+0x270/0x270 [sunrpc]
 rpc_sleep_on_timeout+0x85/0x90 [sunrpc]
 rpc_exit_task+0x5b/0x120 [sunrpc]
 __rpc_execute+0x85/0x3c0 [sunrpc]
 rpc_async_schedule+0x29/0x40 [sunrpc]
 process_one_work+0x1a7/0x360
 worker_thread+0x30/0x390
 ? create_worker+0x1a0/0x1a0
 kthread+0x112/0x130
 ? kthread_flush_work_fn+0x10/0x10
 ret_from_fork+0x35/0x40
....
DEADMAN: Time expired, initiating system panic.
Kernel panic - not syncing: PANIC: Timer expired!
CPU: 11 PID: 0 Comm: swapper/11 Kdump: loaded Tainted: P        W  OEL   --------- -  - 4.18.0-240.15.1.el8_3.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Call Trace:
 <IRQ>
 dump_stack+0x5c/0x80
 panic+0xe7/0x2a9
 ? root_disk_timer_routine+0x25/0x25 [deadman]
 deadman_timer_fire+0x34/0x51 [deadman]
 call_timer_fn+0x2d/0x130
 run_timer_softirq+0x1d8/0x410
 ? enqueue_hrtimer+0x38/0x90
 ? __hrtimer_run_queues+0x130/0x280
 __do_softirq+0xe4/0x2f8