Why am I unable to start my MRG Messaging broker due to the error message " Error dequeing message, persistence id not set"?
Issue
-
MRG cluster disconnected, cannot restart it in cluster mode
-
We were running openais to connect 2 servers for clustering of MRG. One got this series of messages:
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 error Unexpected exception: Error dequeing message, persistence id not set (MessageStoreImpl.cpp:1443)
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 error 10.10.1.181:19775(READY/error) connection error 495440 on 10.10.1.181:19775-389(local): Error dequeing message, persistence id not set (MessageStoreImpl.cpp:1443) (unresolved: 10.10.1.181:19775 10.10.1.182:3607 )
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 error Connection 10.10.1.223:56579 closed by error: Error dequeing message, persistence id not set (MessageStoreImpl.cpp:1443)(501)
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 critical 10.10.1.181:19775(READY/error) error 495440 did not occur on 10.10.1.182:3607
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 error Error delivering frames: Error 495440 did not occur on all members (qpid/cluster/ErrorCheck.cpp:90)
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 notice 10.10.1.181:19775(LEFT/error) leaving cluster ZFS
Jan 28 18:28:03 mq qpidd[19775]: 2010-jan-28 18:28:03 notice Shut down
Jan 28 18:32:14 mq qpidd[23652]: 2010-jan-28 18:32:14 notice Journal "TplStore": Created
Jan 28 18:32:14 mq qpidd[23652]: 2010-jan-28 18:32:14 notice Store module initialized; dir=/var/lib/qpidd
Jan 28 18:32:14 mq qpidd[23652]: 2010-jan-28 18:32:14 notice 10.10.1.181:23652(INIT) membership change: 10.10.1.181:23652 10.10.1.182:3607(left) (joined: 10.10.1.181:23652(joined) )
Jan 28 18:32:14 mq qpidd[23652]: 2010-jan-28 18:32:14 notice Cluster recovery: recovered journal data discarded and journal files pushed down
Jan 28 18:32:14 mq kernel: qpidd[23652]: segfault at 0000000000000001 rip 00002b678f9ee99c rsp 00007fff835589e0 error 4
- When trying to restart the server, it had some status, then a segfault. Only disabling the cluster configuration allowed it to start.
- When attempting to start qpidd, the following messages are seen:
# su - qpidd
-bash-3.2$ /usr/sbin/qpidd --daemon --pid-dir /var/run/qpidd --data-dir /var/lib/qpidd --auth no
Daemon startup failed: illegal-argument: Not enough data for field table. (qpid/framing/FieldTable.cpp:209)-bash-3.2$ /usr/sbin/qpidd --pid-dir /var/run/qpidd --data-dir /var/lib/qpidd --auth no
2010-feb-03 13:59:04 notice Journal "TplStore": Created
2010-feb-03 13:59:04 notice Store module initialized; dir=/var/lib/qpidd
2010-feb-03 13:59:04 notice Journal "6": Created
2010-feb-03
13:59:04 critical Broker start-up failed: illegal-argument: Not enough
data for field table. (qpid/framing/FieldTable.cpp:209)# tail /var/log/messages
Feb 3 14:01:45 mq qpidd[11478]: 2010-feb-03 14:01:45 notice Journal "TplStore": Created
Feb 3 14:01:45 mq qpidd[11478]: 2010-feb-03 14:01:45 notice Store module initialized; dir=/var/lib/qpidd
Feb 3 14:01:45 mq qpidd[11478]: 2010-feb-03 14:01:45 notice 10.10.1.181:11478(INIT) membership change: 10.10.1.181:11478 10.10.1.182:6041(left) (joined: 10.10.1.181:11478(joined) )
Feb 3 14:01:45 mq qpidd[11478]: 2010-feb-03 14:01:45 notice Cluster recovery: recovered journal data discarded and journal files pushed down
Feb 3 14:01:45 mq kernel: qpidd[11478]: segfault at 0000000000000001 rip 00002b8ec0b1f99c rsp 00007fff354beff0 error 4
Feb 3 14:01:45 mq qpidd[11477]: 2010-feb-03 14:01:45 critical Broker start-up failed: Cannot read from child process.
Feb 3 14:01:54 mq kernel: qpidd[11485]: segfault at 0000000000000001 rip 00002b610e58599c rsp 00007fff64976cf0 error 4
Feb 3 14:09:14 mq kernel: qpidd[11497]: segfault at 0000000000000001 rip 00002b2deadc799c rsp 00007fff67f67a50 error 4
Environment
- Red Hat Enterprise Linux 5 update 4
- Red Hat MRG Messaging 1.2
- qpidd-0.5.752581-34.el5-x86_64
- openais-0.80.6-8.el5_4.1
- qpidd-cluster-0.5.752581-34.el5
- rhm-0.5.3206-9.el5
- Clustered qpid configuration
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.