qpid broker fails to start on store recovery: with error "Cannot allocate memory"

Solution Verified - Updated -

Issue

  • after stopping qpid broker, unable to start it again
  • several particular error messages seen, most often with Cannot allocate memory:
    • Error on recovery (MessageStoreImpl.cpp:701): Dbc::get: Cannot allocate memory
    • JERR__AIO: AIO error. (io_queue_init() failed: errno=12 (Cannot allocate memory)
    • JERR_JDIR_STAT: File stat failed. (file="/var/lib/qpidd/.qpidd/qls/jrnl/pulp.agent.ea302561-219c-42a7-bd18-2e3038349769" errno=2 (No such file or directory)
    • JERR__MALLOC: Buffer memory allocation failed. (posix_memalign(): alignment=4096 size=1048576 errno=12 (Cannot allocate memory))
  • the machine has much memory that is not used
  • ulimit or kernel tunables tuned appropriatelly
  • after wiping out provisioning (under /var/lib/qpidd/ directory), broker can start (but without the stored provisioning and journals - unacceptable solution)

Environment

  • Red Hat Enterprise Linux (RHEL) 6 or 7
  • Red Hat Enterprise MRG Messaging
    • store module used - both legacy store (qpid-cpp-server-store) and linear store (qpid-cpp-server-linearstore) could be affected

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.