RHEL7.4: nfsd crashes in __locks_delete_block called from nfs4_state_shutdown_net

Solution In Progress - Updated -

Issue

  • NFS server (nfsd) crashes in __locks_delete_block called from nfs4_state_shutdown_net and nfsd_destroy with oops message as follows
[3714448.523031] IP: [<ffffffff81256cbf>] __locks_delete_block+0x1f/0x60
[3714448.523110] PGD 19f5067 PUD 19f7067 PMD 81c37b067 PTE 816e0a161
[3714448.523187] Oops: 0003 [#1] SMP
[3714448.523249] Modules linked in: rpcsec_gss_krb5 nfsv3 nfs fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 vmw_vmci shpchp parport_pc parport nfsd nfs_acl lockd auth_rpcgss binfmt_misc grace sunrpc ip_tables
[3714448.524083]  xfs libcrc32c sr_mod cdrom ata_generic pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod drm ata_piix crc_t10dif crct10dif_generic ahci libahci libata i2c_core crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
[3714448.524461] CPU: 0 PID: 1575 Comm: nfsd Not tainted 3.10.0-693.11.1.el7.x86_64 #1
[3714448.524586] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
[3714448.524724] task: ffff88081c05eeb0 ti: ffff8808067d4000 task.ti: ffff8808067d4000
[3714448.524846] RIP: 0010:[<ffffffff81256cbf>]  [<ffffffff81256cbf>] __locks_delete_block+0x1f/0x60
[3714448.524980] RSP: 0018:ffff8808067d7d90  EFLAGS: 00010282
[3714448.525050] RAX: ffffffffc0392e50 RBX: ffff88081c2848d0 RCX: dead000000000200
[3714448.525171] RDX: ffff88081c2848d0 RSI: ffff8808067d7dd8 RDI: ffff88081c2848d0
[3714448.525293] RBP: ffff8808067d7da0 R08: ffff88081c2848b8 R09: 4640000000000000
[3714448.525415] R10: dfdf1766f90848c8 R11: ffffea001e362740 R12: ffff88081c2848d0
[3714448.525537] R13: ffff88081c284800 R14: ffff88081c2848a8 R15: ffff88081c2848b8
[3714448.525660] FS:  0000000000000000(0000) GS:ffff88083fc00000(0000) knlGS:0000000000000000
[3714448.525787] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[3714448.526918] CR2: ffffffffc0392e50 CR3: 00000000361ed000 CR4: 00000000000407f0
[3714448.527055] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[3714448.527194] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[3714448.527315] Stack:
[3714448.527370]  ffff88081c2848d0 ffff88081c2848d0 ffff8808067d7db8 ffffffff81256ea7
[3714448.527497]  ffff8808067d7dd8 ffff8808067d7e18 ffffffffc0396989 ffff88081c28494c
[3714448.527623]  ffffffff81ad9d40 ffff8804be7ea810 ffff8804be7ea810 000000005071b149
[3714448.527750] Call Trace:
[3714448.527810]  [<ffffffff81256ea7>] posix_unblock_lock+0x27/0x50
[3714448.527896]  [<ffffffffc0396989>] nfs4_state_shutdown_net+0x1d9/0x330 [nfsd]
[3714448.528024]  [<ffffffffc036a7f8>] nfsd_shutdown_net+0x38/0x70 [nfsd]
[3714448.528103]  [<ffffffffc036a8c9>] nfsd_last_thread+0x99/0xc0 [nfsd]
[3714448.528188]  [<ffffffffc0325ffe>] svc_shutdown_net+0x2e/0x40 [sunrpc]
[3714448.528268]  [<ffffffffc036adfb>] nfsd_destroy+0x5b/0x80 [nfsd]
[3714448.528345]  [<ffffffffc036af43>] nfsd+0x123/0x150 [nfsd]
[3714448.528420]  [<ffffffffc036ae20>] ? nfsd_destroy+0x80/0x80 [nfsd]
[3714448.528496]  [<ffffffff810b099f>] kthread+0xcf/0xe0
[3714448.528566]  [<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[3714448.528643]  [<ffffffff816b4fd8>] ret_from_fork+0x58/0x90
[3714448.528715]  [<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[3714448.528790] Code: ff 5b 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 54 53 48 8b 47 10 48 89 fb 48 85 c0 74 20 48 8b 57 08 48 85 d2 <48> 89 10 74 04 48 89 42 08 48 c7 43 08 00 00 00 00 48 c7 43 10
[3714448.529069] RIP  [<ffffffff81256cbf>] __locks_delete_block+0x1f/0x60
[3714448.529148]  RSP <ffff8808067d7d90>
[3714448.529211] CR2: ffffffffc0392e50

Environment

  • Red Hat Enterprise Linux 7.4 (NFS server)
    • kernel-3.10.0-693.*.el7 or above

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.