A gfs2 filesystem had a withdraw: function = gfs2_meta_indirect_buffer, file = fs/gfs2/meta_io.c, line = 428
Issue
A gfs2 filesystem experienced the below withdraw:
Apr 2 13:52:11 node42 kernel: GFS2: fsid=mycluster:shared_data.0: fatal: invalid metadata block
Apr 2 13:52:11 node42 kernel: GFS2: fsid=mycluster:shared_data.0: bh = 16777216 (magic number)
Apr 2 13:52:11 node42 kernel: GFS2: fsid=mycluster:shared_data.0: function = gfs2_meta_indirect_buffer, file = fs/gfs2/meta_io.c, line = 428
Apr 2 13:52:11 node42 kernel: GFS2: fsid=mycluster:shared_data.0: about to withdraw this file system
Apr 2 13:52:11 node42 kernel: GFS2: fsid=mycluster:shared_data.0: telling LM to unmount
Apr 2 13:52:12 node42 kernel: GFS2: fsid=mycluster:shared_data.0: withdrawn
Apr 2 13:52:12 node42 kernel: CPU: 2 PID: 35139 Comm: oracle Kdump: loaded Tainted: G W OE ------------ 3.10.0-1062.18.1.el7.x86_64 #1
Apr 2 13:52:12 node42 kernel: Hardware name: HP ProLiant DL380p Gen8, BIOS P70 05/24/2019
Apr 2 13:52:12 node42 kernel: Call Trace:
Apr 2 13:52:12 node42 kernel: [<ffffffff97f7b416>] dump_stack+0x19/0x1b
Apr 2 13:52:12 node42 kernel: [<ffffffffc0cbfe06>] gfs2_lm_withdraw+0x146/0x180 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0cc0125>] gfs2_meta_check_ii+0x45/0x50 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6a69>] gfs2_meta_indirect_buffer+0xe9/0x150 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c8fc34>] __fillup_metapath+0x44/0x90 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c8fcba>] fillup_metapath+0x3a/0x50 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c901fe>] gfs2_hole_size.isra.26+0x2ce/0x3a0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c907d0>] gfs2_iomap_get+0x500/0x520 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c932e8>] gfs2_block_map+0x1a8/0x290 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c90>] gfs2_get_block_direct+0x10/0x20 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97a8e6a6>] do_blockdev_direct_IO+0xf96/0x20a0
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c80>] ? gfs2_meta_ra+0x1b0/0x1b0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97a8f805>] __blockdev_direct_IO+0x55/0x60
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c80>] ? gfs2_meta_ra+0x1b0/0x1b0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca717f>] gfs2_direct_IO+0x1df/0x220 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c80>] ? gfs2_meta_ra+0x1b0/0x1b0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca7011>] ? gfs2_direct_IO+0x71/0x220 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff979bfad3>] generic_file_direct_write+0xd3/0x190
Apr 2 13:52:12 node42 kernel: [<ffffffff979bfdc7>] __generic_file_aio_write+0x237/0x400
Apr 2 13:52:12 node42 kernel: [<ffffffff979bffe9>] generic_file_aio_write+0x59/0xa0
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca9bf1>] gfs2_file_aio_write+0xe1/0x310 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97a4dac8>] ? __sb_start_write+0x58/0x120
Apr 2 13:52:12 node42 kernel: [<ffffffff97a4bd7e>] ? do_readv_writev+0x19e/0x260
Apr 2 13:52:12 node42 kernel: [<ffffffff97b04757>] ? security_file_permission+0x27/0xa0
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca9b10>] ? gfs2_release+0x50/0x50 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97aa1ce3>] do_io_submit+0x3e3/0x8a0
Apr 2 13:52:12 node42 kernel: [<ffffffff97aa21b0>] SyS_io_submit+0x10/0x20
Apr 2 13:52:12 node42 kernel: [<ffffffff97f8dede>] system_call_fastpath+0x25/0x2a
Apr 2 13:52:12 node42 kernel: [<ffffffff97f8de21>] ? system_call_after_swapgs+0xae/0x146
Apr 2 13:52:12 node42 kernel: ------------[ cut here ]------------
Apr 2 13:52:12 node42 kernel: WARNING: CPU: 2 PID: 35139 at fs/buffer.c:1166 __brelse+0x2e/0x50
Apr 2 13:52:12 node42 kernel: VFS: brelse: Trying to free free buffer
Apr 2 13:52:12 node42 kernel: Modules linked in: gfs2 dlm mptctl mptbase ib_core scsi_transport_iscsi bridge stp llc ext4 mbcache jbd2 iTCO_wdt iTCO_vendor_support sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw joydev gf128mul glue_helper ablk_helper cryptd pcspkr hpwdt lpc_ich hpilo ioatdma dca ses enclosure scsi_transport_sas ipmi_si ipmi_devintf wmi pcc_cpufreq ipmi_msghandler sg acpi_power_meter binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi dm_round_robin sd_mod crc_t10dif crct10dif_generic uas usb_storage mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt qla2xxx(OE) fb_sys_fops ttm ahci drm ata_piix libahci crct10dif_pclmul crct10dif_common crc32c_intel libata nvme_fc
Apr 2 13:52:12 node42 kernel: serio_raw dm_multipath nvme_fabrics nvme_core tg3(OE) scsi_transport_fc ptp drm_panel_orientation_quirks pps_core scsi_tgt dm_mirror dm_region_hash dm_log dm_mod
Apr 2 13:52:12 node42 kernel: CPU: 2 PID: 35139 Comm: oracle Kdump: loaded Tainted: G W OE ------------ 3.10.0-1062.18.1.el7.x86_64 #1
Apr 2 13:52:12 node42 kernel: Hardware name: HP ProLiant DL380p Gen8, BIOS P70 05/24/2019
Apr 2 13:52:12 node42 kernel: Call Trace:
Apr 2 13:52:12 node42 kernel: [<ffffffff97f7b416>] dump_stack+0x19/0x1b
Apr 2 13:52:12 node42 kernel: [<ffffffff9789bab8>] __warn+0xd8/0x100
Apr 2 13:52:12 node42 kernel: [<ffffffff9789bb3f>] warn_slowpath_fmt+0x5f/0x80
Apr 2 13:52:12 node42 kernel: [<ffffffff97a8269e>] __brelse+0x2e/0x50
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c8fcf6>] release_metapath+0x26/0x40 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c90183>] gfs2_hole_size.isra.26+0x253/0x3a0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c907d0>] gfs2_iomap_get+0x500/0x520 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0c932e8>] gfs2_block_map+0x1a8/0x290 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c90>] gfs2_get_block_direct+0x10/0x20 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97a8e6a6>] do_blockdev_direct_IO+0xf96/0x20a0
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c80>] ? gfs2_meta_ra+0x1b0/0x1b0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97a8f805>] __blockdev_direct_IO+0x55/0x60
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c80>] ? gfs2_meta_ra+0x1b0/0x1b0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca717f>] gfs2_direct_IO+0x1df/0x220 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca6c80>] ? gfs2_meta_ra+0x1b0/0x1b0 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca7011>] ? gfs2_direct_IO+0x71/0x220 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff979bfad3>] generic_file_direct_write+0xd3/0x190
Apr 2 13:52:12 node42 kernel: [<ffffffff979bfdc7>] __generic_file_aio_write+0x237/0x400
Apr 2 13:52:12 node42 kernel: [<ffffffff979bffe9>] generic_file_aio_write+0x59/0xa0
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca9bf1>] gfs2_file_aio_write+0xe1/0x310 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97a4dac8>] ? __sb_start_write+0x58/0x120
Apr 2 13:52:12 node42 kernel: [<ffffffff97a4bd7e>] ? do_readv_writev+0x19e/0x260
Apr 2 13:52:12 node42 kernel: [<ffffffff97b04757>] ? security_file_permission+0x27/0xa0
Apr 2 13:52:12 node42 kernel: [<ffffffffc0ca9b10>] ? gfs2_release+0x50/0x50 [gfs2]
Apr 2 13:52:12 node42 kernel: [<ffffffff97aa1ce3>] do_io_submit+0x3e3/0x8a0
Apr 2 13:52:12 node42 kernel: [<ffffffff97aa21b0>] SyS_io_submit+0x10/0x20
Apr 2 13:52:12 node42 kernel: [<ffffffff97f8dede>] system_call_fastpath+0x25/0x2a
Apr 2 13:52:12 node42 kernel: [<ffffffff97f8de21>] ? system_call_after_swapgs+0xae/0x146
Apr 2 13:52:12 node42 kernel: ---[ end trace 09d699ce5bfc0f6d ]---
Environment
- Red Hat Enterprise Linux Server 7 or 8 (with the High Availability and Resilient Storage Add Ons)
- A Global Filesystem 2 (
gfs2
) - Pacemaker
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.