Stuck messages and CPU churn when aborted transacted message expires

Issue

If a message expires while a transacted consumer rolls back the transaction involving that message, the message is correctly copied to ActiveMQ.DLQ but it does not get cleared from its original destination. As a result it hangs on the original destination, cannot be browsed or consumed.
A broker restart eventually clears this message on the next expiry check but raises this warning on the DLQ:

Duplicate message add attempt rejected. Destination: ActiveMQ.DLQ, Message id: ID:mac.fritz.box-63739-1409131146099-1:1:1:1:10