Kernel panics at fscache_put_operation+472 under heavy NFS load

Solution Unverified - Updated -

Issue

  • Kernel panics at fscache_put_operation+472 under heavy NFS load
  • The following kernel log messages are observed at the time of the occurrence of the kernel panic.
[ 9610.374053] FS-Cache: Assertion failed
[ 9610.374103] 3 == 5 is false
[ 9610.374187] ------------[ cut here ]------------
[ 9610.374239] kernel BUG at fs/fscache/operation.c:419!
[ 9610.374290] invalid opcode: 0000 [#1] SMP 
[ 9610.374338] Modules linked in: fuse btrfs zlib_deflate raid6_pq xor vfat msdos fat ext4 mbcache jbd2 binfmt_misc sg rpcsec_gss_krb5 nfsv4 dns_resolver nfs ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables cachefiles fscache coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd serio_raw vmw_vmci pcspkr parport_pc parport i2c_piix4 shpchp mperf nfsd auth_rpcgss nfs_acl
[ 9610.375471]  lockd sunrpc uinput xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_common vmwgfx ttm drm ata_piix i2c_core vmxnet3 libata vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
[ 9610.375591] CPU: 2 PID: 25251 Comm: kworker/u16:4 Not tainted 3.10.0-123.4.4.el7.x86_64 #1
[ 9610.375626] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 06/22/2012
[ 9610.375678] Workqueue: fscache_operation fscache_op_work_func [fscache]
[ 9610.375708] task: ffff8804274ba220 ti: ffff8801a5c30000 task.ti: ffff8801a5c30000
[ 9610.375739] RIP: 0010:[<ffffffffa03c6a48>]  [<ffffffffa03c6a48>] fscache_put_operation+0x1d8/0x210 [fscache]
[ 9610.375786] RSP: 0018:ffff8801a5c31de8  EFLAGS: 00010246
[ 9610.375810] RAX: 000000000000000f RBX: ffff880428778900 RCX: 0000000000000006
[ 9610.375840] RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000246
[ 9610.375870] RBP: ffff8801a5c31e00 R08: 0000000000000086 R09: 00000000000005a1
[ 9610.375900] R10: 0000000000000082 R11: 0000000000000000 R12: ffff88042365e580
[ 9610.375929] R13: ffff8804295b3800 R14: ffff880425078000 R15: 0000000000000200
[ 9610.375960] FS:  0000000000000000(0000) GS:ffff88043fc80000(0000) knlGS:0000000000000000
[ 9610.377088] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9610.377115] CR2: 00007f3b9e6cb000 CR3: 0000000425f20000 CR4: 00000000000407e0
[ 9610.377184] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9610.377249] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9610.377280] Stack:
[ 9610.377293]  ffff880428778900 ffff88042365e580 ffff8804295b3800 ffff8801a5c31e18
[ 9610.377331]  ffffffffa03c7b0a ffff880428778900 ffff8801a5c31e60 ffffffff8107e03b
[ 9610.377368]  00000000295b3818 0000000000000000 ffff8804295b3818 ffff88042365e5b0
[ 9610.377403] Call Trace:
[ 9610.377428]  [<ffffffffa03c7b0a>] fscache_op_work_func+0x2a/0x50 [fscache]
[ 9610.377464]  [<ffffffff8107e03b>] process_one_work+0x17b/0x460
[ 9610.377493]  [<ffffffff8107ee0b>] worker_thread+0x11b/0x400
[ 9610.377520]  [<ffffffff8107ecf0>] ? rescuer_thread+0x400/0x400
[ 9610.377554]  [<ffffffff81085aff>] kthread+0xcf/0xe0
[ 9610.377579]  [<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
[ 9610.377611]  [<ffffffff815f252c>] ret_from_fork+0x7c/0xb0
[ 9610.377905]  [<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
[ 9610.377935] Code: 31 c0 e8 03 4b 21 e1 48 c7 c7 e5 b3 3c a0 31 c0 e8 f5 4a 21 e1 8b 73 40 ba 05 00 00 00 48 c7 c7 30 b4 3c a0 31 c0 e8 df 4a 21 e1 <0f> 0b 48 c7 c7 e1 b3 3c a0 31 c0 e8 cf 4a 21 e1 48 c7 c7 e5 b3 
[ 9610.426984] RIP  [<ffffffffa03c6a48>] fscache_put_operation+0x1d8/0x210 [fscache]
[ 9610.428154]  RSP <ffff8801a5c31de8>

Environment

  • Red Hat Enterprise Linux 7 (3.10.0-123.4.4.el7.x86_64)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.