A kernel panic occurred on a RHEL 6 or RHEL 7 clustered node with gfs2 mount: "kernel BUG at fs/gfs2/rgrp.c:645!" or " kernel BUG at fs/gfs2/rgrp.c:649!"

Issue

• A kernel panic occurred on a RHEL 6 clustered node with gfs2 mounts:

------------[ cut here ]------------
kernel BUG at fs/gfs2/rgrp.c:645!
invalid opcode: 0000 [#1] SMP 
last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:06:00.1/host3/rport-3:0-9/target3:0:9/3:0:9:11/state
CPU 10 
Modules linked in: gfs2 dlm configfs mptctl mptbase nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc pcc_cpufreq arpt_mangle arptable_filter arp_tables ipv6 xfs exportfs microcode iTCO_wdt iTCO_vendor_support joydev serio_raw lpc_ich mfd_core hpilo hpwdt i7core_edac edac_core ipmi_devintf sg power_meter acpi_ipmi ipmi_si ipmi_msghandler shpchp ext4 jbd2 mbcache dm_round_robin sd_mod qla2xxx lpfc scsi_transport_fc scsi_tgt crc_t10dif be2net dm_multipath dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]

Pid: 27880, comm: nfsd Not tainted 2.6.32-642.1.1.el6.x86_64 #1 HP ProLiant BL460c G7
RIP: 0010:[<ffffffffa075bbe6>]  [<ffffffffa075bbe6>] gfs2_rsqa_delete+0x76/0x80 [gfs2]
RSP: 0018:ffff8817b31dfc60  EFLAGS: 00010202
RAX: 0000000000000020 RBX: ffff882d00784be0 RCX: 0000000000000000
RDX: ffff882fd5d34ea8 RSI: ffff882fd5d34f20 RDI: ffff882d00784f18
RBP: ffff8817b31dfc80 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000002000 R11: 0000000000000000 R12: ffff882d00784e0c
R13: ffff882d00784f68 R14: ffff8822b4866e00 R15: ffff8817d12e5980
FS:  0000000000000000(0000) GS:ffff8818608a0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007f317a150000 CR3: 0000000001a8d000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process nfsd (pid: 27880, threadinfo ffff8817b31dc000, task ffff8817e21d0040)
Stack:
 ffff882f09ef77c0 ffff882d00784be0 ffff882f09ef77c0 ffff882d00784be0
<d> ffff8817b31dfca0 ffffffffa074f30b ffff882f09ef77c0 0000000000000008
<d> ffff8817b31dfcf0 ffffffff8119b7d5 ffff882d00784be0 ffff8822b4866e00
Call Trace:
 [<ffffffffa074f30b>] gfs2_release+0x4b/0x60 [gfs2]
 [<ffffffff8119b7d5>] __fput+0xf5/0x210
 [<ffffffff8119b915>] fput+0x25/0x30
 [<ffffffffa067c343>] nfsd_write+0xf3/0x100 [nfsd]
 [<ffffffffa0684d5f>] nfsd3_proc_write+0xaf/0x140 [nfsd]
 [<ffffffffa0675405>] nfsd_dispatch+0xe5/0x230 [nfsd]
 [<ffffffffa055bd04>] svc_process_common+0x344/0x640 [sunrpc]
 [<ffffffff8106c720>] ? default_wake_function+0x0/0x20
 [<ffffffffa055c3a0>] svc_process+0x110/0x160 [sunrpc]
 [<ffffffffa0675c62>] nfsd+0xc2/0x160 [nfsd]
 [<ffffffffa0675ba0>] ? nfsd+0x0/0x160 [nfsd]
 [<ffffffff810a662e>] kthread+0x9e/0xc0
 [<ffffffff8100c28a>] child_rip+0xa/0x20
 [<ffffffff810a6590>] ? kthread+0x0/0xc0
 [<ffffffff8100c280>] ? child_rip+0x0/0x20
Code: 91 b4 ff ff 48 8b 5d e8 4c 8b 65 f0 4c 8b 6d f8 c9 c3 0f 1f 00 48 8d bb f8 02 00 00 e8 44 ff ff ff 8b 83 68 03 00 00 85 c0 74 c6 <0f> 0b eb fe 66 0f 1f 44 00 00 55 48 89 e5 41 56 41 55 41 54 53 
RIP  [<ffffffffa075bbe6>] gfs2_rsqa_delete+0x76/0x80 [gfs2]
 RSP <ffff8817b31dfc60>

• A kernel panic occurred on a RHEL 7 clustered node with gfs2 mounts:

[65473.060807] ------------[ cut here ]------------
[65473.065549] kernel BUG at fs/gfs2/rgrp.c:649!
[65473.070015] invalid opcode: 0000 [#1] SMP
[65473.074244] Modules linked in: gfs2 dlm mpt2sas raid_class mptctl mptbase xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ext4 mbcache jbd2 intel_powerclamp coretemp kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper iTCO_wdt ablk_helper cryptd iTCO_vendor_support dcdbas ipmi_devintf i7core_edac ses ipmi_ssif pcspkr enclosure dm_service_time lpc_ich scsi_transport_sas edac_core ipmi_si mfd_core sg shpchp ipmi_msghandler acpi_power_meter wmi acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc dm_multipath ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic i2c_algo_bit ixgbe drm_kms_helper mdio syscopyarea sysfillrect ptp sysimgblt fb_sys_fops crct10dif_pclmul qla2xxx pps_core ttm scsi_transport_fc ata_piix drm crct10dif_common crc32c_intel libata serio_raw i2c_core scsi_tgt dca megaraid_sas bnx2 fjes dm_mirror dm_region_hash dm_log dm_mod
[65473.176056] CPU: 22 PID: 2901 Comm: kworker/22:0 Not tainted 3.10.0-443.el7.x86_64 #1
[65473.184107] Hardware name: Dell Inc. PowerEdge R910/0KYD3D, BIOS 2.10.0 08/29/2013
[65473.191916] Workqueue: events delayed_fput
[65473.196128] task: ffff8818023a8ed0 ti: ffff88178231c000 task.ti: ffff88178231c000
[65473.203840] RIP: 0010:[<ffffffffa08b3389>]  [<ffffffffa08b3389>] gfs2_rsqa_delete+0x69/0x70 [gfs2]
[65473.213056] RSP: 0018:ffff88178231fd60  EFLAGS: 00010202
[65473.218477] RAX: 00000000000000c8 RBX: ffff88046ea97578 RCX: 0000000000000000
[65473.225721] RDX: ffff88066eb5a330 RSI: ffff88066eb5a3b0 RDI: ffff88066eb5a3ac
[65473.232969] RBP: ffff88178231fd78 R08: 0000000000000000 R09: 00000001802a0024
[65473.240213] R10: 000000000131d901 R11: ffffea006004c740 R12: ffff88046ea97440
[65473.247453] R13: ffff88046ea977c0 R14: ffff8817a00e2600 R15: ffff88178ccf1220
[65473.254699] FS:  0000000000000000(0000) GS:ffff88185f940000(0000) knlGS:0000000000000000
[65473.263016] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[65473.268871] CR2: 00007fe454727044 CR3: 000000204e72b000 CR4: 00000000000007e0
[65473.276112] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[65473.283357] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[65473.290599] Stack:
[65473.292716]  ffff8817c8cd9d00 ffff88046ea97440 ffff88046ea97440 ffff88178231fd98
[65473.300371]  ffffffffa08a50ff ffff8817c8cd9d00 0000000000000008 ffff88178231fde0
[65473.308034]  ffffffff811f5c39 ffff88046ea97440 ffff8817c8cd9d10 ffff8817c8cd9d00
[65473.315693] Call Trace:
[65473.318255]  [<ffffffffa08a50ff>] gfs2_release+0x3f/0x50 [gfs2]
[65473.324281]  [<ffffffff811f5c39>] __fput+0xe9/0x260
[65473.329269]  [<ffffffff811f5e5f>] delayed_fput+0xaf/0x130
[65473.334779]  [<ffffffff810a577b>] process_one_work+0x17b/0x470
[65473.340722]  [<ffffffff810a65b6>] worker_thread+0x126/0x410
[65473.346402]  [<ffffffff810a6490>] ? rescuer_thread+0x460/0x460
[65473.354963]  [<ffffffff810add6f>] kthread+0xcf/0xe0
[65473.359952]  [<ffffffff810adca0>] ? kthread_create_on_node+0x140/0x140
[65473.366590]  [<ffffffff8167ded8>] ret_from_fork+0x58/0x90
[65473.372102]  [<ffffffff810adca0>] ? kthread_create_on_node+0x140/0x140
[65473.378738] Code: 3e bf ff ff 5b 41 5c 41 5d 5d c3 0f 1f 80 00 00 00 00 49 8d bc 24 f0 02 00 00 e8 63 ff ff ff 41 8b 84 24 60 03 00 00 85 c0 74 c6 <0f> 0b 0f 1f 44 00 00 66 66 66 66 90 55 48 89 e5 41 56 4c 8d b7
[65473.398554] RIP  [<ffffffffa08b3389>] gfs2_rsqa_delete+0x69/0x70 [gfs2]