- Red Hat Enterprise Linux 6.3
- We are using Red Hat Enterprise Linux with Cluster function on Oracle 11G. When switching the Oracle service over from node A to node B, the following error messages were logged in "/var/log/message":
Oct 8 16:05:15 node2 rgmanager: [fs] mounting /dev/dm-28 on /share Oct 8 16:05:15 node2 rgmanager: [fs] mount -t ext4 /dev/dm-28 /share Oct 8 16:05:15 node2 kernel: EXT4-fs (dm-28): bad geometry: block count 130809856 exceeds size of device (12845056 blocks) Oct 8 16:05:16 node2 rgmanager: [fs] 'mount -t ext4 /dev/dm-28 /share' failed, error=32 Oct 8 16:05:19 node2 rgmanager: start on fs "shared_fs" returned 1 (generic error) Oct 8 16:05:19 node2 rgmanager: #68: Failed to start service:sharedfs; return value: 1 Oct 8 16:05:19 node2 rgmanager: Stopping service service:sharedfs
1. Disable the cluster service on both cluster nodes by taking scheduled downtime, create a data backup from of the filesystem, ensure the LVM/Filesystem resources are unmounted from both cluster nodes and that these shares are not being used anywhere. Perform a filesystem(e2fsck) check on the filesystem to clean it.
2. It appears that an incorrect HA LVM configuration lead to this issue. Correct the HA LVM configuration on this cluster should resolve the issue. Please refer following article which outlines the steps to implement HA LVM. There are two ways to do this, using clvmd method or using LVM Tagging.
3. Enable the self_fence option for filesystem resource to ensure that the service is still able to relocate by first rebooting the original node.
The service failed since it could not mount the cluster filesystem resource which is required for cluster service.
Following messages were also logged on another cluster node which suggests to perform a filesystem check:
Oct 8 16:17:27 node1 rgmanager: [fs] mount -t ext4 /dev/dm-34 /data1 Oct 8 16:17:27 node1 kernel: EXT4-fs (dm-34): warning: maximal mount count reached, running e2fsck is recommended Oct 8 16:17:27 node1 kernel: EXT4-fs (dm-34): mounted filesystem with ordered data mode. Opts: Oct 8 16:17:27 node1 rgmanager: [fs] mount -t ext4 /dev/dm-36 /data2 Oct 8 16:17:27 node1 kernel: EXT4-fs (dm-36): warning: maximal mount count reached, running e2fsck is recommended Oct 8 16:17:27 node1 kernel: EXT4-fs (dm-36): mounted filesystem with ordered data mode. Opts:
- It is recommended that all filesystem resources(fs.sh) are created on a HA-LVM device. Make sure the HA-LVM has been configured in cluster.
- Red Hat Enterprise Linux
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.