RHEL Clustering
Hi,
I was finally able to get the correct subscriptions setup and have all the clustering prerequisites now installed.
The only cluster I am after is an application cluster, this will consist of 3 applications:
- httpd
- app1
- app2
They need to be started/stopped in a specific order:
Startup Sequence:
- app1
- app2
- httpd
Stop Sequence:
- httpd
- app2
- app1
I am struggling with my cluster.conf to accomplish this. All the above have /etc/init.d/ scripts which obviously return 0 for success and > 0 for not.
The other thing I am having trouble with is the 2 nodes fencing each other. The fencing works at the VMware level and indeed works, however, when a node comes in and joins the cluster it always seems to fence the other node which is running the service and forcefully restarts on itself. Correct me if I'm wrong, but I don't want this to occur if the service is running correctly.
I can't seem to find any valid application cluster.conf examples around. Does anyone do something like the above and care to share their cluster.conf?
Thanks,
Terry
Responses
Terry,
From your post, I could see that you have multiple issues? I believe, your cluster config needs to be validated by Red Hat Global Support Services cluster specialists team. Is it possible for you to open a case with Red Hat GSS. The reason I am asking to contact them is that they can verify your cluster logs and can eliminate possible issues. Cluster specialists will review your cluster configs and OS environment then will provide you their recommendations.
Regards,
Anand
Terry,
Thanks for the clarification. I hope your account issues will be sort out soon.
The nodes fencing each other also called split-brain issue. By own each cluster node can form its own cluster, which leads to split-brain.
There are multiple ways to prevent fence races in 2-node clusters:
- Use a shared fence device that only allows one log in at a time (such as an APC or WTI PDU).
- Use the 'fence delay attribute' to allow one node to always win races (RHEL 5 Update 6 or later, RHEL 6 Update 1 or later)
- Use a quorum device
You may able to find more details on above topic in Red Hat customer portal. My suggestion is to test fencing config first before starting services. If everything works as expected then move on to configuring services.
I hope It helps.
Regards,
Anand
The best reference for getting resources to start and stop in the order you want is in one of the appendicies in the Cluster Administration guide:
https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/Cluster_Administration/index.html#ap-ha-resource-behavior-CA
* Since each of your resources are the same type (script), they should start and stop based on the order that they appear in /etc/cluster/cluster.conf. Resources are started in a serial fashion; one does not start until the previous one has finished starting, and the same goes for stopping.
* If you want to enforce the order you described, you can use parent-child relationships, something like this inside your <service/>
<script name="httpd" file="/etc/init.d/httpd">
<script name="app2" file="/etc/init.d/app2">
<script name="app1" file="/etc/init.d/app1"/>
</script>
</script>
* Some applications take a while to get going. Make sure that when your scripts 'exit 0' that the application is totally up in terms of whatever the next resource will need, and build in delays if needed.
* rg_test is an excellent tool for debugging issues with service start. Make sure to disable the service before experimenting with rg_test:
How can I debug a clustered service in Cluster Suite for Red Hat Enterprise Linux 4, 5, or 6?
=> https://access.redhat.com/knowledge/solutions/3994
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.