GFS2 cluster nodes often crashe due to soft lockups that happen in gfs2_dir_get_hash_table()

Issue

• GFS2 cluster nodes often crash due to soft lockups happen in gfs2_dir_get_hash_table():

[2145038.728625] Kernel panic - not syncing: softlockup: hung tasks
[2145038.728629] CPU: 7 PID: 27628 Comm: http-nio-8000-e Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.81.1.el7.x86_64 #1
[2145038.728630] c3 cc 
[2145038.728630] 
[2145038.728633] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
[2145038.728634] Call Trace:
[2145038.728636] 90 <8b> 17 85 d2 75 
[2145038.728637]  <IRQ> 
[2145038.728640]  [<ffffffffa05b1bec>] dump_stack+0x19/0x1f
[2145038.728645]  [<ffffffffa05ab708>] panic+0xe8/0x21f
[2145038.728651]  [<ffffffff9ff56f1a>] watchdog_timer_fn+0x20a/0x220
[2145038.728656]  [<ffffffff9ff56d10>] ? watchdog+0x50/0x50
[2145038.728660]  [<ffffffff9fecf9ce>] __hrtimer_run_queues+0x10e/0x270
[2145038.728669]  [<ffffffff9fecff5f>] hrtimer_interrupt+0xaf/0x1e0
[2145038.728674]  [<ffffffff9fe5fa2b>] local_apic_timer_interrupt+0x3b/0x70
[2145038.728675] f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 cc 
[2145038.728680]  [<ffffffffa05ca053>] smp_apic_timer_interrupt+0x43/0x60
[2145038.728684]  [<ffffffffa05c63f2>] apic_timer_interrupt+0x172/0x180
[2145038.728690]  <EOI>  [<ffffffff9ff1e97d>] ? native_queued_spin_lock_slowpath+0x1d/0x200
[2145038.728694]  [<ffffffffa05ac21a>] queued_spin_lock_slowpath+0xb/0x13
[2145038.728698]  [<ffffffffa05ba734>] _raw_spin_lock+0x24/0x30
[2145038.728721]  [<ffffffffc0817678>] gfs2_dir_get_hash_table+0x2a8/0x460 [gfs2]
[2145038.728746]  [<ffffffffc08171c0>] ? gfs2_dirent_prev+0x70/0x70 [gfs2]
[2145038.728768]  [<ffffffffc0817847>] get_leaf_nr+0x17/0x40 [gfs2]
[2145038.728792]  [<ffffffffc0817926>] gfs2_dirent_search+0xb6/0x1c0 [gfs2]
[2145038.728815]  [<ffffffffc08191ec>] gfs2_dir_check+0x3c/0xf0 [gfs2]
[2145038.728836]  [<ffffffffc082c882>] gfs2_drevalidate+0x132/0x1f0 [gfs2]
[2145038.728858]  [<ffffffffc082c85d>] ? gfs2_drevalidate+0x10d/0x1f0 [gfs2]
[2145038.728863]  [<ffffffffa0067eda>] lookup_fast+0x1da/0x230
[2145038.728867]  [<ffffffffa011a872>] ? security_inode_permission+0x22/0x30
[2145038.728872]  [<ffffffffa0069fd3>] link_path_walk+0x1b3/0x8b0
[2145038.728877]  [<ffffffffa006a84a>] path_lookupat+0x7a/0x8d0
[2145038.728900]  [<ffffffffc08207e5>] ? gfs2_holder_wake+0x25/0x60 [gfs2]
[2145038.728904]  [<ffffffffa00668e7>] ? path_get+0x27/0x30
[2145038.728907]  [<ffffffffa006b0cb>] filename_lookup+0x2b/0xd0
[2145038.728911]  [<ffffffffa006ee47>] user_path_at_empty+0x67/0xc0
[2145038.728916]  [<ffffffffa012de40>] ? sidtab_search+0x10/0x20
[2145038.728920]  [<ffffffffa0139f9f>] ? selinux_audit_rule_match+0x6f/0x300
[2145038.728924]  [<ffffffffa006eeb1>] user_path_at+0x11/0x20
[2145038.728927]  [<ffffffffa00614e3>] vfs_fstatat+0x63/0xd0
[2145038.728932]  [<ffffffffa00618de>] SYSC_newstat+0x2e/0x60
[2145038.728936]  [<ffffffffa0061dee>] SyS_newstat+0xe/0x20
[2145038.728940]  [<ffffffffa05c539a>] system_call_fastpath+0x25/0x2a