RHEL7: System crashed with RIP: __fscache_disable_cookie during NFS umount

Solution In Progress - Updated -

Issue

  • System crashed during NFS unmount in context of __fscache_disable_cookie call
[3891892.449375] FS-Cache: 
[3891892.449409] FS-Cache: Assertion failed
[3891892.449425] FS-Cache: 0 > 0 is false
[3891892.449508] ------------[ cut here ]------------
[3891892.449526] kernel BUG at fs/fscache/cookie.c:519!
crash> bt
PID: 7533   TASK: ffff880b96345e20  CPU: 4   COMMAND: "umount.nfs4"
 #0 [ffff880b9b7939c0] machine_kexec at ffffffff81059bab
 #1 [ffff880b9b793a20] __crash_kexec at ffffffff81105822
 #2 [ffff880b9b793af0] crash_kexec at ffffffff81105910
 #3 [ffff880b9b793b08] oops_end at ffffffff81690008
 #4 [ffff880b9b793b30] die at ffffffff8102e93b
 #5 [ffff880b9b793b60] do_trap at ffffffff8168f6c0
 #6 [ffff880b9b793bb0] do_invalid_op at ffffffff8102b144
 #7 [ffff880b9b793c60] invalid_op at ffffffff8169909e
    [exception RIP: __fscache_disable_cookie+456]
    RIP: ffffffffa04cd0e8  RSP: ffff880b9b793d10  RFLAGS: 00010246
    RAX: 0000000000000018  RBX: ffff880b4637d528  RCX: 0000000000000006
    RDX: 0000000000000000  RSI: 0000000000000246  RDI: ffff880c3fd0f830
    RBP: ffff880b9b793d38   R8: 0000000000000082   R9: 0000000000000b89
    R10: 0000000000000001  R11: 0000000000000000  R12: ffff880b4637d528
    R13: ffffffffa05a57c0  R14: ffff880b9b793e00  R15: ffff880c05b1bba0
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #8 [ffff880b9b793d40] __fscache_relinquish_cookie at ffffffffa04cd629 [fscache]
 #9 [ffff880b9b793d58] nfs_fscache_clear_inode at ffffffffa0548afb [nfs]
#10 [ffff880b9b793d78] nfs_clear_inode at ffffffffa053298f [nfs]
#11 [ffff880b9b793d90] nfs4_evict_inode at ffffffffa058b331 [nfsv4]
#12 [ffff880b9b793da8] evict at ffffffff8121aa67
#13 [ffff880b9b793dd0] dispose_list at ffffffff8121ab6e
#14 [ffff880b9b793df8] evict_inodes at ffffffff8121b7c4
#15 [ffff880b9b793e48] generic_shutdown_super at ffffffff81200ff8
#16 [ffff880b9b793e70] kill_anon_super at ffffffff812013f2
#17 [ffff880b9b793e88] nfs_kill_super at ffffffffa0537abb [nfs]
#18 [ffff880b9b793ea8] deactivate_locked_super at ffffffff812017a9
#19 [ffff880b9b793ec8] deactivate_super at ffffffff81201da6
#20 [ffff880b9b793ee0] mntput_no_expire at ffffffff8121f1e5
#21 [ffff880b9b793f08] sys_umount at ffffffff81220320
#22 [ffff880b9b793f80] system_call_fastpath at ffffffff81697809
    RIP: 00007ff5a714c807  RSP: 00007ffceb95acb8  RFLAGS: 00010206
    RAX: 00000000000000a6  RBX: ffffffff81697809  RCX: 0000000000000010
    RDX: 0000000000000001  RSI: 0000000000000000  RDI: 00007ff5a7edd4e0
    RBP: 00007ff5a7edd4e0   R8: 61735f7472697673   R9: 69665f786f62646e
    R10: 00007ffceb95a9a0  R11: 0000000000000246  R12: 00007ffceb95c84f
    R13: 00007ff5a7eddea0  R14: 0000000000000000  R15: 00007ff5a7655d38
    ORIG_RAX: 00000000000000a6  CS: 0033  SS: 002b

Example 2: Another stack trace

crash> bt 
PID: 22428  TASK: ffff8829942c5ee0  CPU: 12  COMMAND: "sshd"
 #0 [ffff883071ed7750] machine_kexec at ffffffff8105c4cb
 #1 [ffff883071ed77b0] __crash_kexec at ffffffff81104a32
 #2 [ffff883071ed7880] crash_kexec at ffffffff81104b20
 #3 [ffff883071ed7898] oops_end at ffffffff816ad278
 #4 [ffff883071ed78c0] die at ffffffff8102e97b
 #5 [ffff883071ed78f0] do_trap at ffffffff816ac9c0
 #6 [ffff883071ed7940] do_invalid_op at ffffffff8102b184
 #7 [ffff883071ed79f0] invalid_op at ffffffff816b685e
    [exception RIP: fscache_object_destroy+0x37]
    RIP: ffffffffc10d8e17  RSP: ffff883071ed7aa0  RFLAGS: 00010246
    RAX: ffff880790422001  RBX: ffff881ce3fef000  RCX: 00000000005e0afd
    RDX: 0000000000000000  RSI: ffff883fed7e4300  RDI: ffff8828caa20aa8
    RBP: ffff883071ed7aa8   R8: 0000000000019b80   R9: ffffffffc111bfdf
    R10: ffff883fff099b80  R11: ffffea001e410880  R12: ffff881ff98ad000
    R13: ffff8828caa20aa8  R14: ffff881ce3fef000  R15: ffff883a1922bf30
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #8 [ffff883071ed7ab0] cachefiles_put_object at ffffffffc111bff6 [cachefiles]
 #9 [ffff883071ed7ad0] fscache_alloc_object at ffffffffc10d7aab [fscache]
#10 [ffff883071ed7b18] fscache_alloc_object at ffffffffc10d79a6 [fscache]
#11 [ffff883071ed7b60] fscache_acquire_non_index_cookie at ffffffffc10d81ac [fscache]
#12 [ffff883071ed7ba0] __fscache_enable_cookie at ffffffffc10d8398 [fscache]
#13 [ffff883071ed7bd0] nfs_fscache_open_file at ffffffffc15220f2 [nfs]
#14 [ffff883071ed7bf0] nfs4_file_open at ffffffffc1568185 [nfsv4]
#15 [ffff883071ed7ca0] do_dentry_open at ffffffff811fe667
#16 [ffff883071ed7ce8] vfs_open at ffffffff811fe83a
#17 [ffff883071ed7d10] do_last at ffffffff8120f88d
#18 [ffff883071ed7db0] path_openat at ffffffff81210a22
#19 [ffff883071ed7e48] do_filp_open at ffffffff81212f4b
#20 [ffff883071ed7f18] do_sys_open at ffffffff811ffc03
#21 [ffff883071ed7f70] sys_open at ffffffff811ffd1e
#22 [ffff883071ed7f80] system_call_fastpath at ffffffff816b4fc9

Environment

  • Red Hat Enterprise Linux 7
  • NFS version 4

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content