qpid broker fails to start on store recovery: with error "Cannot allocate memory"
Issue
- after stopping
qpid
broker, unable to start it again - several particular error messages seen, most often with
Cannot allocate memory
:Error on recovery (MessageStoreImpl.cpp:701): Dbc::get: Cannot allocate memory
JERR__AIO: AIO error. (io_queue_init() failed: errno=12 (Cannot allocate memory)
JERR_JDIR_STAT: File stat failed. (file="/var/lib/qpidd/.qpidd/qls/jrnl/pulp.agent.ea302561-219c-42a7-bd18-2e3038349769" errno=2 (No such file or directory)
JERR__MALLOC: Buffer memory allocation failed. (posix_memalign(): alignment=4096 size=1048576 errno=12 (Cannot allocate memory))
- the machine has much memory that is not used
- ulimit or kernel tunables tuned appropriatelly
- after wiping out provisioning (under
/var/lib/qpidd/
directory), broker can start (but without the stored provisioning and journals - unacceptable solution)
Environment
- Red Hat Enterprise Linux (RHEL) 6 or 7
- Red Hat Enterprise MRG Messaging
- store module used - both legacy store (
qpid-cpp-server-store
) and linear store (qpid-cpp-server-linearstore
) could be affected
- store module used - both legacy store (
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.