cman dumps core file when starting on Red Hat Enterprise Linux 5
Issue
- When starting cman service, cman is dumping a core file and the cluster does not form.
-
The following error messages are seen in
/var/log/messagesafter the core file is dumped:openais[26509]: [CLM ] CLM CONFIGURATION CHANGE openais[26509]: [CLM ] New Configuration: openais[26509]: [CLM ] r(0) ip(127.0.0.1) openais[26509]: [CLM ] Members Left: openais[26509]: [CLM ] Members Joined: openais[26509]: [CLM ] CLM CONFIGURATION CHANGE openais[26509]: [CLM ] New Configuration: openais[26509]: [CLM ] r(0) ip(127.0.0.1) openais[26509]: [CLM ] Members Left: openais[26509]: [CLM ] Members Joined: openais[26509]: [SYNC ] This node is within the primary component and will provide service. openais[26509]: [TOTEM] entering OPERATIONAL state. openais[26509]: [CLM ] got nodejoin message (null) openais[26509]: [EVT ] recovery error node: r(0) ip(127.0.0.1) not found fenced[26532]: cluster is down, exiting gfs_controld[26544]: cluster is down, exiting dlm_controld[26538]: cluster is down, exiting -
The following backtrace is observed from the core file when analyzed with gdb:
(gdb) bt #0 0x0000003e60c30265 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x0000003e60c31d10 in abort () at abort.c:88 #2 0x0000003e60c296e6 in __assert_fail (assertion=<value optimized out>, file=<value optimized out>, line=<value optimized out>, function=<value optimized out>) at assert.c:78 #3 0x00002aaaabb281b7 in evt_sync_process () at evt.c:4152 #4 0x000000000041a586 in sync_service_process (type=<value optimized out>, data=0x733690) at sync.c:237 #5 0x000000000040e51a in token_callbacks_execute (instance=<value optimized out>, type=<value optimized out>) at totemsrp.c:3052 #6 0x0000000000411e3a in message_handler_orf_token (instance=0x2aaaaaaae010, msg=<value optimized out>, msg_len=<value optimized out>, endian_conversion_needed=<value optimized out>) at totemsrp.c:3438 #7 0x0000000000409f43 in rrp_deliver_fn (context=0x2aaaac018370, msg=0x2aaaac018a58, msg_len=70) at totemrrp.c:1308 #8 0x00000000004084fb in net_deliver_fn (handle=<value optimized out>, fd=<value optimized out>, revents=<value optimized out>, data=0x2aaaac0183b0) at totemnet.c:695 #9 0x0000000000405d10 in poll_run (handle=0) at aispoll.c:402 #10 0x0000000000418834 in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:620
Environment
- Red Hat Enterprise Linux 5.4
- Red Hat Cluster Suite.
- 2 node cluster configuration
- Private network for heartbeat
- Cluster package levels:
- openais-0.80.6-8.el5
- cman-2.0.115-34.el5_5.3
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.