Master node shutdown / hangs while performing purge operation on a large queue

Issue

• AMQ purge operations results in the following log by the Critical Analyzer.

2019-08-30 12:08:45,474 WARN  [org.apache.activemq.artemis.utils.critical.CriticalMeasure] Component org.apache.activemq.artemis.core.server.impl.QueueImpl is expired on path 2                                                                                                                                              
2019-08-30 12:08:45,477 ERROR [org.apache.activemq.artemis.core.server] AMQ224079: The process for the virtual machine will be killed, as component QueueImpl[name=6e70fa78-2c5d-49d8-b7c8-51f9b42ce44f, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5a85c170-b535-11e9-9ce1-7cd30ada465a], temp=true]@7a│

• AMQ message purge operation takes longer than expected and seems to use more memory with a large queue size. Sometimes it triggers an OutOfMemoryException (OOME) The thread stack shows this is triggered by a blocking thread invoking removeAllMessages()

• The maaster broker shutdown while doing purge operations and failed over to the slave. Thread dump shows a number of blocked threads
  - blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@7a3be658

With the lock being held by

"xyz-85160" Id=85168 RUNNABLE
    at sun.nio.ch.NativeThread.current(Native Method)
    at sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
    at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:155)
    -  locked java.lang.Object@50d09802
    at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.read(NIOSequentialFile.java:163)
    -  locked org.apache.activemq.artemis.core.io.nio.NIOSequentialFile@43b7e70b
    at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.read(NIOSequentialFile.java:153)
    at org.apache.activemq.artemis.core.paging.impl.Page.readIntoFileBufferIfNecessary(Page.java:158)
    at org.apache.activemq.artemis.core.paging.impl.Page.readFromSequentialFile(Page.java:218)
    at org.apache.activemq.artemis.core.paging.impl.Page.read(Page.java:114)
    -  locked org.apache.activemq.artemis.core.paging.impl.Page@141a3e8a
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.readPage(PageCursorProviderImpl.java:212)
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.getPageCache(PageCursorProviderImpl.java:175)
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.getMessage(PageCursorProviderImpl.java:129)
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl.getReference(PageSubscriptionImpl.java:369)
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl.access$2000(PageSubscriptionImpl.java:63)
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.moveNext(PageSubscriptionImpl.java:1304)
    -  locked java.util.LinkedList@6c5bcb2a
    -  locked org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl@1b36d6fe
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.next(PageSubscriptionImpl.java:1284)
    at org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.hasNext(PageSubscriptionImpl.java:1408)
    -  locked org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator@381f5e2b
    at org.apache.activemq.artemis.core.server.impl.QueueImpl.iterQueue(QueueImpl.java:1866)
    -  locked org.apache.activemq.artemis.core.server.impl.QueueImpl@7a3be658
    at org.apache.activemq.artemis.core.server.impl.QueueImpl.deleteMatchingReferences(QueueImpl.java:1785)
    -  locked org.apache.activemq.artemis.core.server.impl.QueueImpl@7a3be658
    at org.apache.activemq.artemis.core.server.Queue.deleteMatchingReferences(Queue.java:263)
    at org.apache.activemq.artemis.core.management.impl.QueueControlImpl.removeMessages(QueueControlImpl.java:976)
    at org.apache.activemq.artemis.core.management.impl.QueueControlImpl.removeAllMessages(QueueControlImpl.java:984)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)