18.17. Performance Tuning
18.17.1. Tuning Persistence
- Put the message journal on its own physical volume. If the disk is shared with other processes, for example transaction co-ordinator, database or other journals, which are also reading and writing from it, then this may greatly reduce performance since the disk head may be skipping between the different files. One of the advantages of an append only journal is that disk head movement is minimized. This advantage is lost if the disk is shared. If you are using paging or large messages, make sure they are put on separate volumes too.
- Minimum number of journal files. Set
journal-min-filesparameter to a number of files that would fit your average sustainable rate. If you see new files being created on the journal data directory too often, that is, lots of data is being persisted, you need to increase the minimal number of files, this way the journal would reuse more files instead of creating new data files.
- Journal file size. The journal file size must be aligned to the capacity of a cylinder on the disk. The default value of 10MiB should be enough on most systems.
- Use AIO journal. For Linux operating system, keep your journal type as
AIOwill scale better than Java NIO.
journal-buffer-timeout. The timeout can be increased to increase throughput at the expense of latency.
- If you are running AIO you might be able to get improved performance by increasing
journal-max-ioparameter value. Do not change this parameter if you are running NIO.
18.17.2. Tuning JMS
- Disable message ID. Use the
setDisableMessageID()method on the
MessageProducerclass to disable message IDs if you do not need them. This decreases the size of the message and also avoids the overhead of creating a unique ID.
- Disable message timestamp. Use the
setDisableMessageTimeStamp()method on the
MessageProducerclass to disable message timestamps if you do not need them.
ObjectMessageis convenient but it comes at a cost. The body of a
ObjectMessageuses Java serialization to serialize it to bytes. The Java serialized form of even small objects is very verbose so takes up a lot of space on the wire, also Java serialization is slow compared to custom marshalling techniques. Only use
ObjectMessageif you really cannot use one of the other message types, that is if you do not know the type of the payload until run-time.
AUTO_ACKNOWLEDGEmode requires an acknowledgement to be sent from the server for each message received on the client, this means more traffic on the network. If you can, use
CLIENT_ACKNOWLEDGEor a transacted session and batch up many acknowledgements with one acknowledge/commit.
- Avoid durable messages. By default, JMS messages are durable. If you do not need durable messages then set them to be
non-durable. Durable messages incur a lot more overhead in persisting them to storage.
- Batch many sends or acknowledgements in a single transaction. HornetQ will only require a network round trip on the commit, not on every send or acknowledgement.
18.17.3. Other Tunings
- Use Asynchronous Send Acknowledgements. If you need to send durable messages non transactional and you need a guarantee that they have reached the server by the time the call to
send()returns, do not set durable messages to be sent blocking, instead use asynchronous send acknowledgements to get your acknowledgements of send back in a separate stream.
pre-acknowledgemode, messages are acknowledged before they are sent to the client. This reduces the amount of acknowledgement traffic on the wire.
- Disable security. There is a small performance boost when you disable security by setting the
security-enabledparameter to false in
- Disable persistence. You can turn off message persistence altogether by setting
persistence-enabledto false in
- Sync transactions lazily. Setting
journal-sync-transactionalto false in
domain.xmlgives better transactional persistent performance at the expense of some possibility of loss of transactions on failure.
- Sync non transactional lazily. Setting
journal-sync-non-transactionalto false in
domain.xmlgives better non-transactional persistent performance at the expense of some possibility of loss of durable messages on failure
- Send messages non blocking. Setting
block-on-non-durable-sendto false in
domain.xml(if you are using JMS and JNDI) or directly on the ServerLocator. This means you do not have to wait a whole network round trip for every message sent.
- If you have very fast consumers, you can increase
consumer-window-size. This effectively disables consumer flow control.
- Socket NIO vs Socket Old IO. By default HornetQ uses old (blocking) on the server and the client side. NIO is much more scalable but can give some latency hit compared to old blocking IO. To service many thousands of connections on the server, then you must use NIO on the server. However, if there are no thousands of connections on the server you can keep the server acceptors using old IO, and you may get a small performance advantage.
- Use the core API not JMS. Using the JMS API you will have slightly lower performance than using the core API, since all JMS operations need to be translated into core operations before the server can handle them. If using the core API try to use methods that take
SimpleStringas much as possible.
java.lang.Stringdoes not require copying before it is written to the wire, so if you re-use
SimpleStringinstances between calls then you can avoid some unnecessary copying.
18.17.4. Tuning Transport Settings
- TCP buffer sizes. If you have a fast network and fast machines you may get a performance boost by increasing the TCP send and receive buffer sizes.
NoteSome operating systems like later versions of Linux include TCP auto-tuning and setting TCP buffer sizes manually can prevent auto-tune from working and actually give you worse performance.
- Increase limit on file handles on the server. If you expect a lot of concurrent connections on your servers, or if clients are rapidly opening and closing connections, you must make sure the user running the server has permission to create sufficient file handles.This varies from operating system to operating system. On Linux systems you can increase the number of allowable open file handles in the file
/etc/security/limits.conf. For example, add the lines
serveruser soft nofile 20000 serveruser hard nofile 20000This would allow up to 20000 file handles to be open by the user
direct-deliverto false for the best throughput for very small messages. HornetQ comes with a preconfigured connector/acceptor pair
domain.xmland JMS connection factory
domain.xmlwhich can be used to give the very best throughput, especially for small messages.
18.17.5. Tuning the VM
- Garbage collection. For smooth server operation, it is recommended to use a parallel garbage collection algorithm. For example, using the JVM argument
-XX:+UseParallelGCon Sun JDKs.
- Memory settings. Give as much memory as you can to the server. HornetQ can run in low memory by using paging but if it can run with all queues in RAM this will improve performance. The amount of memory you require will depend on the size and number of your queues and the size and number of your messages. Use the JVM arguments
-Xmxto set server available RAM. We recommend setting them to the same high value.
- Aggressive options. Different JVMs provide different sets of JVM tuning parameters. It is recommended at least using
-XX:+UseFastAccessorMethods. You may get some mileage with the other tuning parameters depending on your operating system platform and application usage patterns.
18.17.6. Avoiding Anti-Patterns
- Re-use connections / sessions / consumers / producers. Probably the most common messaging anti-pattern we see is users who create a new connection/session/producer for every message they send or every message they consume. This is a poor use of resources. These objects take time to create and may involve several network round trips. Always re-use them.
NoteSome popular libraries such as the Spring JMS Template use these anti-patterns. If you are using Spring JMS Template, you may get poor performance. The Spring JMS Template can only safely be used in an application server which caches JMS sessions, example, using JCA), and only then for sending messages. It cannot be safely be used for synchronously consuming messages, even in an application server.
- Avoid fat messages. Verbose formats such as XML take up a lot of space on the wire and performance will suffer as result. Avoid XML in message bodies if you can.
- Do not create temporary queues for each request. This common anti-pattern involves the temporary queue request-response pattern. With the temporary queue request-response pattern a message is sent to a target and a reply-to header is set with the address of a local temporary queue. When the recipient receives the message they process it then send back a response to the address specified in the reply-to. A common mistake made with this pattern is to create a new temporary queue on each message sent. This drastically reduces performance. Instead the temporary queue should be re-used for many requests.
- Do not use Message-Driven Beans for the sake of it. As soon as you start using MDBs you are greatly increasing the codepath for each message received compared to a straightforward message consumer, since a lot of extra application server code is executed.