Soft lockup occurs in the NFS delegation recovery code

Solution Unverified - Updated -

Issue

  • Soft lockup occurs with below log:

    [1724834.494631] NMI watchdog: BUG: soft lockup - CPU#60 stuck for 23s! [192.168.1.1-man:38898]
    [1724834.494637] Modules linked in: tcp_lp fuse rpcsec_gss_krb5 nfsv4 nfs fscache vmw_vsock_vmci_transport vsock ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport nfit libnvdimm vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl rbd lockd grace libceph sunrpc dns_resolver binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif pata_acpi crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw i2c_core vmxnet3 vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
    [1724834.494716] CPU: 60 PID: 38898 Comm: 192.168.1.1-man Kdump: loaded Not tainted 3.10.0-862.el7.x86_64 #1
    [1724834.494718] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
    [1724834.494721] task: ffff8b2b74f19fa0 ti: ffff8b2a1d22c000 task.ti: ffff8b2a1d22c000
    [1724834.494724] RIP: 0010:[<ffffffff94115670>]  [<ffffffff94115670>] _raw_spin_lock+0x10/0x30
    [1724834.494736] RSP: 0018:ffff8b2a1d22fd70  EFLAGS: 00000246
    [1724834.494738] RAX: 0000000000000000 RBX: 0000000000000246 RCX: ffff8b7fdd570000
    [1724834.494740] RDX: 0000000000000001 RSI: 0000000000000007 RDI: ffff8ad2bb2a1ef8
    [1724834.494742] RBP: ffff8b2a1d22fd88 R08: ffff8b7fdd57488c R09: ffffffff93b969c7
    [1724834.494744] R10: ffff8b7fe225bac0 R11: ffffd9af7008d980 R12: 0000000000001400
    [1724834.494746] R13: ffff8b7fd9c4e548 R14: 0000000000000000 R15: ffffffff93ab1391
    [1724834.494748] FS:  0000000000000000(0000) GS:ffff8b7fe2240000(0000) knlGS:0000000000000000
    [1724834.494749] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [1724834.494751] CR2: 00007f75939c6000 CR3: 000000606f7ce000 CR4: 00000000003607e0
    [1724834.494798] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    [1724834.494800] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
    [1724834.494801] Call Trace:
    [1724834.494809]  [<ffffffff93c37a8e>] ? igrab+0x1e/0x60
    [1724834.494828]  [<ffffffffc07f69aa>] nfs_delegation_grab_inode+0x2a/0x50 [nfsv4]
    [1724834.494837]  [<ffffffffc07f7d2c>] nfs_delegation_reap_unclaimed+0xac/0x140 [nfsv4]
    [1724834.494844]  [<ffffffffc07f2038>] nfs4_state_clear_reclaim_reboot+0x138/0x160 [nfsv4]
    [1724834.494850]  [<ffffffffc07f4170>] nfs4_state_manager+0x5d0/0x8d0 [nfsv4]
    [1724834.494856]  [<ffffffffc07f4470>] ? nfs4_state_manager+0x8d0/0x8d0 [nfsv4]
    [1724834.494861]  [<ffffffffc07f448f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
    [1724834.494869]  [<ffffffff93abae31>] kthread+0xd1/0xe0
    [1724834.494871]  [<ffffffff93abad60>] ? insert_kthread_work+0x40/0x40
    [1724834.494875]  [<ffffffff9411f637>] ret_from_fork_nospec_begin+0x21/0x21
    [1724834.494878]  [<ffffffff93abad60>] ? insert_kthread_work+0x40/0x40
    [1724834.494879] Code: 44 00 00 85 d2 74 e4 0f 1f 40 00 eb ed 66 0f 1f 44 00 00 b8 01 00 00 00 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 <85> c0 75 01 c3 55 89 c6 48 89 e5 e8 9f 2d ff ff 5d c3 0f 1f 40 
    
  • [<NFS server's IP address>-manager] kernel thread consumes 100% CPU resource.

Environment

  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 7
  • NFSv4 client

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content