Why did my GFS2 filesystem withdraw with 'function = gfs2_check_internal_file_size' on RHEL 6?

Issue

• Why did my GFS2 filesystem withdraw with function = gfs2_check_internal_file_size on RHEL 6?
• A withdraw will occur if a cluster node tries to mount a GFS2 filesystem in spectator mode when there is no other cluster nodes with the GFS2 filesystem mounted in non-spectator mode.

Aug 16 09:59:44 example.com kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "rh6nodesThree:target1lvol1"
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: Joined cluster. Now mounting FS...
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: jid=0, already locked for use
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: jid=0: Looking at journal...
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: fatal: filesystem consistency error
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s:   inode = 4 25
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s:   function = gfs2_check_internal_file_size, file = fs/gfs2/inode.h, line = 93
Aug 16 09:59:44 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: about to withdraw this file system
Aug 16 09:59:45 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: telling LM to unmount
Aug 16 09:59:45 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: withdrawn
Aug 16 09:59:45 example.com kernel: Pid: 1813, comm: kslowd001 Not tainted 2.6.32-358.2.1.el6.x86_64 #1
Aug 16 09:59:45 example.com kernel: Call Trace:
Aug 16 09:59:45 example.com kernel: [<ffffffffa060de32>] ? gfs2_lm_withdraw+0x102/0x130 [gfs2]
Aug 16 09:59:45 example.com kernel: [<ffffffffa060e03d>] ? gfs2_consist_inode_i+0x5d/0x60 [gfs2]
Aug 16 09:59:45 example.com kernel: [<ffffffffa060a8ab>] ? gfs2_jdesc_check+0x6b/0xd0 [gfs2]
Aug 16 09:59:45 example.com kernel: [<ffffffffa0605b0d>] ? gfs2_recover_work+0xed/0x7b0 [gfs2]
Aug 16 09:59:45 example.com kernel: [<ffffffff8105e203>] ? perf_event_task_sched_out+0x33/0x80
Aug 16 09:59:45 example.com kernel: [<ffffffff8106659b>] ? dequeue_task_fair+0x12b/0x130
Aug 16 09:59:45 example.com kernel: [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
Aug 16 09:59:45 example.com kernel: [<ffffffff8150d960>] ? thread_return+0x4e/0x76e
Aug 16 09:59:45 example.com kernel: [<ffffffff81065ceb>] ? enqueue_task_fair+0xfb/0x100
Aug 16 09:59:45 example.com kernel: [<ffffffff8110e233>] ? slow_work_execute+0x233/0x310
Aug 16 09:59:45 example.com kernel: [<ffffffff8110e467>] ? slow_work_thread+0x157/0x360
Aug 16 09:59:45 example.com kernel: [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
Aug 16 09:59:45 example.com kernel: [<ffffffff8110e310>] ? slow_work_thread+0x0/0x360
Aug 16 09:59:45 example.com kernel: [<ffffffff81096936>] ? kthread+0x96/0xa0
Aug 16 09:59:45 example.com kernel: [<ffffffff8100c0ca>] ? child_rip+0xa/0x20
Aug 16 09:59:45 example.com kernel: [<ffffffff810968a0>] ? kthread+0x0/0xa0
Aug 16 09:59:45 example.com kernel: [<ffffffff8100c0c0>] ? child_rip+0x0/0x20
Aug 16 09:59:45 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: jid=0: Failed
Aug 16 09:59:45 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: jid=1: Trying to acquire journal lock...
Aug 16 09:59:45 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: jid=2: Trying to acquire journal lock...
Aug 16 09:59:45 example.com kernel: GFS2: fsid=rh6nodesThree:target1lvol1.s: can't read in statfs inode: -5
Aug 16 09:59:45 example.com kernel: gdlm_unlock 1,0 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 1,1 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 5,805b err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 2,18 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 5,17 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 4,0 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 2,17 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 5,16 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 5,100a3 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 5,19 err=-22
Aug 16 09:59:45 example.com kernel: gdlm_unlock 5,18 err=-22
Aug 16 10:02:13 example.com kernel: INFO: task mount.gfs2:26422 blocked for more than 120 seconds.
Aug 16 10:02:13 example.com kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 16 10:02:13 example.com kernel: mount.gfs2    D 0000000000000000     0 26422  26421 0x00000080