file system resource does not fail its status checks after a file system has failed and remounted read-only in a RHEL 6 Update 6 High Availability cluster with rgmanager

Solution Unverified - Updated -

Issue

  • After updating to RHEL 6 Update 6, when I pull a fiber cable to test the cluster's reaction, our fs resource never fails its status checks even though the file system has remounted read-only.
# tail -f /var/log/messages
Jan 19 10:42:44 rhel6-node1 rgmanager[2836]: Service service:fstest started
[...]
Jan 19 10:43:05 rhel6-node1 multipathd: mpathc: remaining active paths: 0
Jan 19 10:43:31 rhel6-node1 multipathd: mpathc: Disable queueing
Jan 19 10:43:31 rhel6-node1 kernel: Buffer I/O error on device dm-3, logical block 131584
Jan 19 10:43:31 rhel6-node1 kernel: lost page write due to I/O error on dm-3
Jan 19 10:43:31 rhel6-node1 kernel: JBD: I/O error detected when updating journal superblock for dm-3.
Jan 19 10:43:31 rhel6-node1 kernel: Aborting journal on device dm-3.
Jan 19 10:43:31 rhel6-node1 kernel: Buffer I/O error on device dm-3, logical block 131584
Jan 19 10:43:31 rhel6-node1 kernel: lost page write due to I/O error on dm-3
Jan 19 10:43:31 rhel6-node1 kernel: JBD: I/O error detected when updating journal superblock for dm-3.
Jan 19 10:43:39 rhel6-node1 kernel: EXT3-fs (dm-3): error: ext3_journal_start_sb: Detected aborted journal
Jan 19 10:43:39 rhel6-node1 kernel: EXT3-fs (dm-3): error: remounting filesystem read-only

[root@rhel6-node1 ~]# tail -f /var/log/cluster/rgmanager.log
Jan 19 10:41:11 rgmanager Starting changed resources.
Jan 19 10:42:44 rgmanager Starting disabled service service:fstest
Jan 19 10:42:44 rgmanager [fs] mounting /dev/dm-3 on /mnt/testmnt
Jan 19 10:42:44 rgmanager [fs] mount -t ext3  /dev/dm-3 /mnt/testmnt
Jan 19 10:42:44 rgmanager Service service:fstest started
Jan 19 10:42:59 rgmanager [fs] Checking fs "mpathc", Level 20
Jan 19 10:43:09 rgmanager [fs] Checking fs "mpathc", Level 20
Jan 19 10:43:19 rgmanager [fs] Checking fs "mpathc", Level 20
Jan 19 10:43:29 rgmanager [fs] Checking fs "mpathc", Level 20
Jan 19 10:43:39 rgmanager [fs] Checking fs "mpathc", Level 20
Jan 19 10:43:49 rgmanager [fs] Checking fs "mpathc", Level 20
Jan 19 10:43:59 rgmanager [fs] Checking fs "mpathc", Level 20
  • fs write tests during the status checks are incorrectly succeeding after a file system has failed

Environment

  • Red Hat Enterprise Linux (RHEL) 6 Update 6 with the High Availability Add On
  • rgmanager
  • resource-agents releases starting with 3.9.5-12.el6 and prior to [3.9.5-12.el6_6.3](https://rhn.redhat.com/errata/RHBA-2015-0130.html
  • One or more fs, clusterfs, or netfs resource resource agents configured in /etc/cluster/cluster.conf

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.