RHEL7: System crashed with RIP: __fscache_disable_cookie during NFS umount

Solution In Progress - Updated -

Issue

  • System crashed during NFS unmount in context of __fscache_disable_cookie call
[3891892.449375] FS-Cache: 
[3891892.449409] FS-Cache: Assertion failed
[3891892.449425] FS-Cache: 0 > 0 is false
[3891892.449508] ------------[ cut here ]------------
[3891892.449526] kernel BUG at fs/fscache/cookie.c:519!
crash> bt
PID: 7533   TASK: ffff880b96345e20  CPU: 4   COMMAND: "umount.nfs4"
 #0 [ffff880b9b7939c0] machine_kexec at ffffffff81059bab
 #1 [ffff880b9b793a20] __crash_kexec at ffffffff81105822
 #2 [ffff880b9b793af0] crash_kexec at ffffffff81105910
 #3 [ffff880b9b793b08] oops_end at ffffffff81690008
 #4 [ffff880b9b793b30] die at ffffffff8102e93b
 #5 [ffff880b9b793b60] do_trap at ffffffff8168f6c0
 #6 [ffff880b9b793bb0] do_invalid_op at ffffffff8102b144
 #7 [ffff880b9b793c60] invalid_op at ffffffff8169909e
    [exception RIP: __fscache_disable_cookie+456]
    RIP: ffffffffa04cd0e8  RSP: ffff880b9b793d10  RFLAGS: 00010246
    RAX: 0000000000000018  RBX: ffff880b4637d528  RCX: 0000000000000006
    RDX: 0000000000000000  RSI: 0000000000000246  RDI: ffff880c3fd0f830
    RBP: ffff880b9b793d38   R8: 0000000000000082   R9: 0000000000000b89
    R10: 0000000000000001  R11: 0000000000000000  R12: ffff880b4637d528
    R13: ffffffffa05a57c0  R14: ffff880b9b793e00  R15: ffff880c05b1bba0
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #8 [ffff880b9b793d40] __fscache_relinquish_cookie at ffffffffa04cd629 [fscache]
 #9 [ffff880b9b793d58] nfs_fscache_clear_inode at ffffffffa0548afb [nfs]
#10 [ffff880b9b793d78] nfs_clear_inode at ffffffffa053298f [nfs]
#11 [ffff880b9b793d90] nfs4_evict_inode at ffffffffa058b331 [nfsv4]
#12 [ffff880b9b793da8] evict at ffffffff8121aa67
#13 [ffff880b9b793dd0] dispose_list at ffffffff8121ab6e
#14 [ffff880b9b793df8] evict_inodes at ffffffff8121b7c4
#15 [ffff880b9b793e48] generic_shutdown_super at ffffffff81200ff8
#16 [ffff880b9b793e70] kill_anon_super at ffffffff812013f2
#17 [ffff880b9b793e88] nfs_kill_super at ffffffffa0537abb [nfs]
#18 [ffff880b9b793ea8] deactivate_locked_super at ffffffff812017a9
#19 [ffff880b9b793ec8] deactivate_super at ffffffff81201da6
#20 [ffff880b9b793ee0] mntput_no_expire at ffffffff8121f1e5
#21 [ffff880b9b793f08] sys_umount at ffffffff81220320
#22 [ffff880b9b793f80] system_call_fastpath at ffffffff81697809
    RIP: 00007ff5a714c807  RSP: 00007ffceb95acb8  RFLAGS: 00010206
    RAX: 00000000000000a6  RBX: ffffffff81697809  RCX: 0000000000000010
    RDX: 0000000000000001  RSI: 0000000000000000  RDI: 00007ff5a7edd4e0
    RBP: 00007ff5a7edd4e0   R8: 61735f7472697673   R9: 69665f786f62646e
    R10: 00007ffceb95a9a0  R11: 0000000000000246  R12: 00007ffceb95c84f
    R13: 00007ff5a7eddea0  R14: 0000000000000000  R15: 00007ff5a7655d38
    ORIG_RAX: 00000000000000a6  CS: 0033  SS: 002b

Example 2: Another stack trace

crash> bt 
PID: 22428  TASK: ffff8829942c5ee0  CPU: 12  COMMAND: "sshd"
 #0 [ffff883071ed7750] machine_kexec at ffffffff8105c4cb
 #1 [ffff883071ed77b0] __crash_kexec at ffffffff81104a32
 #2 [ffff883071ed7880] crash_kexec at ffffffff81104b20
 #3 [ffff883071ed7898] oops_end at ffffffff816ad278
 #4 [ffff883071ed78c0] die at ffffffff8102e97b
 #5 [ffff883071ed78f0] do_trap at ffffffff816ac9c0
 #6 [ffff883071ed7940] do_invalid_op at ffffffff8102b184
 #7 [ffff883071ed79f0] invalid_op at ffffffff816b685e
    [exception RIP: fscache_object_destroy+0x37]
    RIP: ffffffffc10d8e17  RSP: ffff883071ed7aa0  RFLAGS: 00010246
    RAX: ffff880790422001  RBX: ffff881ce3fef000  RCX: 00000000005e0afd
    RDX: 0000000000000000  RSI: ffff883fed7e4300  RDI: ffff8828caa20aa8
    RBP: ffff883071ed7aa8   R8: 0000000000019b80   R9: ffffffffc111bfdf
    R10: ffff883fff099b80  R11: ffffea001e410880  R12: ffff881ff98ad000
    R13: ffff8828caa20aa8  R14: ffff881ce3fef000  R15: ffff883a1922bf30
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #8 [ffff883071ed7ab0] cachefiles_put_object at ffffffffc111bff6 [cachefiles]
 #9 [ffff883071ed7ad0] fscache_alloc_object at ffffffffc10d7aab [fscache]
#10 [ffff883071ed7b18] fscache_alloc_object at ffffffffc10d79a6 [fscache]
#11 [ffff883071ed7b60] fscache_acquire_non_index_cookie at ffffffffc10d81ac [fscache]
#12 [ffff883071ed7ba0] __fscache_enable_cookie at ffffffffc10d8398 [fscache]
#13 [ffff883071ed7bd0] nfs_fscache_open_file at ffffffffc15220f2 [nfs]
#14 [ffff883071ed7bf0] nfs4_file_open at ffffffffc1568185 [nfsv4]
#15 [ffff883071ed7ca0] do_dentry_open at ffffffff811fe667
#16 [ffff883071ed7ce8] vfs_open at ffffffff811fe83a
#17 [ffff883071ed7d10] do_last at ffffffff8120f88d
#18 [ffff883071ed7db0] path_openat at ffffffff81210a22
#19 [ffff883071ed7e48] do_filp_open at ffffffff81212f4b
#20 [ffff883071ed7f18] do_sys_open at ffffffff811ffc03
#21 [ffff883071ed7f70] sys_open at ffffffff811ffd1e
#22 [ffff883071ed7f80] system_call_fastpath at ffffffff816b4fc9

Environment

  • Red Hat Enterprise Linux 7
  • NFS version 4

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In