[RFE] Possibility to create queue files dynamically and resize queues

Issue

We use MRG Messaging to distribute messages to large number of clients. During the day, we usually send up to 250000 messages to the biggest client. Some of the smaller clients receive only hundreds of messages per day. Moreover, in most cases, the clients are immediately consuming the messages from the queues. However, for business reasons, we need to be able to persists all messages on the broker for the day. To achieve this, we are using persistent messages with flow-to-disk queues and we are creating them big enough to contain a large reserve/margin, because we cannot resize the queue on the run (i.e. for 1000000 messages instead of 250000 messages).

The problem is, that the flow-to-disk queues create all the files used for storing of messages when they are created. Therefore, if we have 1000 clients and each client has his own queue of size 1GB (1000000 messages of the size 1024 bytes), we need in total 1TB of disk space for pour queues and when we configure the broker by creating these queues takes a lot of time.

However, since most of our clients are consuming all the messages immediately and do not receive so many messages per day, most of the disk space allocated for the queues is never used.

It would be great to have following possibility:

1) Use queues with dynamically created files. The queue will for example create just one storage journal on the beginning and create the next file only when the capacity of the first reaches 80%. Or the file can even expand dynamically, as the messages are stored into them.

2) Have a possibility to use SQL storage for storing the messages. (this is more or less alternative solution to point 1)

3) To have the possibility to resize the queues dynamically, for example using the qpid-config tool. Right now, you have to delete the queue and create a new bigger one (and all the messages stored in the queue are deleted).