A panic occurred after gfs2 filesystem had a withdrawal: kernel BUG at fs/gfs2/glock.c:492!

Solution In Progress - Updated -

Issue

  • A panic occurred after gfs2 filesystem had a withdrawal: kernel BUG at fs/gfs2/glock.c:492!
[ 1669.918716] GFS2: fsid=ha_cluster:gfs2fs.0: Error -52 writing to log
[ 1669.918764] GFS2: fsid=ha_cluster:gfs2fs.0: fatal: I/O error
GFS2: fsid=ha_cluster:gfs2fs.0:   block = 66703
GFS2: fsid=ha_cluster:gfs2fs.0:   function = gfs2_ail1_empty_one, file = fs/gfs2/log.c, line = 198
[ 1669.918778] GFS2: fsid=ha_cluster:gfs2fs.0: about to withdraw this file system
[ 1669.927897] GFS2: fsid=ha_cluster:gfs2fs.0: telling LM to unmount
[ 1669.928481] GFS2: fsid=ha_cluster:gfs2fs.0: withdrawn
[.....]
[ 1674.919794] ------------[ cut here ]------------
[ 1674.919810] kernel BUG at fs/gfs2/glock.c:492!
[ 1674.919825] invalid opcode: 0000 [#1] SMP
[ 1674.919841] Modules linked in: gfs2 dlm intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_service_time sg iTCO_wdt iTCO_vendor_support mxm_wmi sb_edac dcdbas ipmi_devintf edac_core pcspkr shpchp mei_me mei lpc_ich ipmi_si ipmi_msghandler acpi_power_meter wmi dm_multipath dm_mod ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit crct10dif_pclmul drm_kms_helper crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops bnx2x ttm drm qla2xxx ahci mdio libahci tg3 libcrc32c libata scsi_transport_fc i2c_core ptp megaraid_sas scsi_tgt pps_core fjes
[ 1674.920107] CPU: 8 PID: 11882 Comm: kworker/8:2H Not tainted 3.10.0-514.el7.x86_64 #1
[ 1674.920131] Hardware name: Dell Inc. PowerEdge R730/0599V5, BIOS 2.1.7 06/16/2016
[ 1674.920158] Workqueue: glock_workqueue glock_work_func [gfs2]
[ 1674.920177] task: ffff880270ec3ec0 ti: ffff880268678000 task.ti: ffff880268678000
[ 1674.920199] RIP: 0010:[<ffffffffa04cc659>]  [<ffffffffa04cc659>] do_xmote+0x1d9/0x290 [gfs2]
[ 1674.920228] RSP: 0018:ffff88026867bd50  EFLAGS: 00010286
[ 1674.920245] RAX: 0000000000000000 RBX: ffff880275c88000 RCX: 0000000000000006
[ 1674.920266] RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff88027670f830
[ 1674.920299] RBP: ffff88026867bda0 R08: 0000000000000092 R09: 0000000000000843
[ 1674.920321] R10: 000000000000027f R11: 0000000000000001 R12: ffff880275c88030
[ 1674.920343] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000000220
[ 1674.920365] FS:  0000000000000000(0000) GS:ffff880276700000(0000) knlGS:0000000000000000
[ 1674.920390] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1674.920408] CR2: 00007f2deb1fb000 CR3: 00000000019ba000 CR4: 00000000001407e0
[ 1674.920430] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1674.920452] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 1674.920474] Stack:
[ 1674.921449]  ffff880276716c40 0000000000000000 ffff880274a58000 ffffffffa04f0ba0
[ 1674.922404]  ffffffff810cec2e ffff880275c88000 0000000000000000 ffff880275c88030
[ 1674.923354]  0000000000000000 0000000000000220 ffff88026867bde0 ffffffffa04cc850
[ 1674.924325] Call Trace:
[ 1674.925275]  [<ffffffff810cec2e>] ? dequeue_task_fair+0x41e/0x660
[ 1674.926267]  [<ffffffffa04cc850>] run_queue+0x140/0x270 [gfs2]
[ 1674.927229]  [<ffffffffa04cc9e1>] glock_work_func+0x61/0x140 [gfs2]
[ 1674.928181]  [<ffffffff810a7f3b>] process_one_work+0x17b/0x470
[ 1674.929142]  [<ffffffff810a8d76>] worker_thread+0x126/0x410
[ 1674.930104]  [<ffffffff810a8c50>] ? rescuer_thread+0x460/0x460
[ 1674.931060]  [<ffffffff810b052f>] kthread+0xcf/0xe0
[ 1674.932006]  [<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[ 1674.932962]  [<ffffffff81696418>] ret_from_fork+0x58/0x90
[ 1674.933910]  [<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[ 1674.934854] Code: 1f 00 f6 43 38 0c 75 09 45 85 ed 0f 84 a4 00 00 00 89 c6 48 c7 c7 27 29 4f a0 31 c0 e8 a7 2d 1b e1 48 89 de 31 ff e8 87 ef ff ff <0f> 0b 0f 1f 44 00 00 4c 8b 35 09 e6 02 00 4d 85 f6 74 1b 49 8b
[ 1674.936888] RIP  [<ffffffffa04cc659>] do_xmote+0x1d9/0x290 [gfs2]
[ 1674.937877]  RSP <ffff88026867bd50>
  • A panic occurs after an IO withdrawal of the gfs2 filesystems because the scsi reservations were revoked for the storage when the cluster node was fenced with fence_scsi.

Environment

  • Red Hat Enterprise Linux (RHEL) 7 with the Resilient Storage Add-On
  • One or more gfs2 filesystems were mounted

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.