Chapter 35. Duplicate Message Detection

HornetQ includes powerful automatic duplicate message detection, filtering out duplicate messages without having to code duplicate detection logic at the application level. This chapter will explain what duplicate detection is, how HornetQ uses it and how and where to configure it.
When sending messages from a client to a server, or indeed from a server to another server, if the target server or connection fails sometime after sending the message, but before the sender receives a response that the send (or commit) was processed successfully then the sender cannot know for sure if the message was sent successfully to the address.
If the target server or connection failed after the send was received and processed but before the response was sent back then the message will have been sent to the address successfully, but if the target server or connection failed before the send was received and finished processing then it will not have been sent to the address successfully. From the senders point of view, it is not possible to distinguish between these two cases.
When the server recovers, this leaves the client in a difficult situation. It knows the target server failed, but it does not know if the last message reached its destination satisfactorily. If it decides to resend the last message, it may result in a duplicate message being sent to the address. If each message was an order or a trade then this could result in the order being fulfilled twice or the trade being doubly booked, which is an undesirable situation.
Sending the message(s) in a transaction is also not the correct solution. If the server or connection fails while the transaction commit is being processed, it is indeterminate whether the transaction was successfully committed or not.
To solve these issues HornetQ provides automatic duplicate messages detection for messages sent to addresses.

35.1. Using Duplicate Detection for Message Sending

Enable duplicate message detection for sent messages: set a special property on the message to a uniquely created value. When the target server receives the message it will check whether that property is set. If it is, the target server will check its memory cache whether a message with the value of the header has already been received. If it has previously received a message with the same value it will ignore the message.

Note

Using duplicate detection to move messages between nodes can give the same once and only once delivery guarantees as when an XA transaction is used to consume messages from source and send them to the target, but with less overhead and much easier configuration than using XA.
If sending messages in a transaction it is not necessary to set the property for every message sent in that transaction, set it once in the transaction. If the server detects a duplicate message for any message in the transaction, it will ignore the entire transaction.
The name of the property set is given by the value of org.hornetq.api.core.HDR_DUPLICATE_DETECTION_ID, which is _HQ_DUPL_ID.
The value of the property can be of type byte[] or SimpleString if using the core API. If using JMS it must be a String, and its value should be unique. An easy way of generating a unique ID is by generating a UUID.
An example of setting the property using the core API follows: 
...     

ClientMessage message = session.createMessage(true);

SimpleString myUniqueID = "This is my unique id";   // Could use a UUID for this

message.setStringProperty(HDR_DUPLICATE_DETECTION_ID, myUniqueID);

...
And Here is an example using the JMS API: 
...     

Message jmsMessage = session.createMessage();

String myUniqueID = "This is my unique id";   // Could use a UUID for this

message.setStringProperty(HDR_DUPLICATE_DETECTION_ID.toString(), myUniqueID);

...