Server crashes with rpc.mountd process at svc_export_put+0x3/0x5b [nfsd] called from svc_expkey_lookup

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 4.6.z 
    • 2.6.9-67.0.22.ELsmp

Issue

  • Server crashed with the following oops

    Unable to handle kernel NULL pointer dereference at virtual address 0000000a
     printing eip: f99a7628
    Modules linked in: mptctl mptbase netconsole netdump nfsd exportfs parport_pc lp parport autofs4 i2c_dev i2c_core nfs lockd nfs_acl sunrpc button battery ac md5 ipv6 uhci_hcd ehci_hcd hw_random e1000 tg3 bonding(U) ata_piix libata sg dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod cciss sd_mod qla2xxx(U) scsi_mod qla2xxx_conf(U)
    EIP:    0060:[<f99a7628>]    Not tainted VLI
    EFLAGS: 00010246   (2.6.9-67.0.22.ELsmp) 
    EIP is at svc_export_put+0x3/0x5b [nfsd]
    Process rpc.mountd (pid: 9379, threadinfo=f25c4000 task=f7d12830)
    Stack: f5ea0140 f99a6e98 f6ae0b00 f99a753b 000000ef 00000008 f99ccf54 00000001 
           f0c67000 f5238840 f25c4f54 f25c4ef0 f99a7205 00000008 fffffffe f426e920 
           f25c4efa f0c67001 f98f0284 f599763c f53ef600 00417192 02cd7fa0 00000004 
    Call Trace:
     [<f99a6e98>] expkey_put+0x3c/0x4e [nfsd]
     [<f99a753b>] svc_expkey_lookup+0x238/0x322 [nfsd]
     [<f99a7205>] expkey_parse+0x2d8/0x332 [nfsd]
     [<c0144a65>] __pagevec_free+0x15/0x1a
     [<c014d9af>] unmap_vmas+0xf1/0x205
     [<f99a6f2d>] expkey_parse+0x0/0x332 [nfsd]
     [<f98daff4>] cache_write+0xf8/0x115 [sunrpc]
     [<c015bfc3>] vfs_write+0xb6/0xe2
     [<c015c08d>] sys_write+0x3c/0x62
     [<c02d8b7b>] syscall_call+0x7/0xb
     [<c02d007b>] xfrm_alloc_userspi+0x132/0x178
    Code: 30 1f 33 c0 ba d0 00 00 00 e8 cf 0e 7a c6 85 c0 89 c7 74 0a e8 42 2d f3 ff e9 9d fd ff ff 31 c0 83 c4 10 5b 5e 5f 5d c3 53 89 c3 <8b> 40 0c 83 f8 02 7f 0b 8b 43 04 3b 42 34 7d 03 89 42 34 f0 ff 
    

Resolution

Root Cause

  • Known race condition between nfsd and rpc.mountd in svc_expkey_lookup() resulting in an excess cache_put() call on any svc_export structure.

Diagnostic Steps

  • Compare oops message, confirm kernel version.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.