Chapter 12. Configuring routing

Routing is the process by which messages are delivered to their destinations. To accomplish this, AMQ Interconnect provides two routing mechanisms: message routing and link routing.

Message routing
Message routing is the default routing mechanism. You can use it to route messages on a per-message basis between clients directly (direct-routed messaging), or to and from broker queues (brokered messaging).
Link routing
A link route represents a private messaging path between a sender and a receiver in which the router passes the messages between end points. You can use it to connect a client to a service (such as a broker queue).

12.1. Configuring message routing

Message routing is the default routing mechanism. You can use it to route messages on a per-message basis between clients directly (direct-routed messaging), or to and from broker queues (brokered messaging).

With message routing, you can do the following:

12.1.1. Understanding message routing

With message routing, routing is performed on messages as producers send them to a router. When a message arrives on a router, the router routes the message and its settlement based on the message’s address and routing pattern.

12.1.1.1. Message routing flow control

AMQ Interconnect uses a credit-based flow control mechanism to ensure that producers can only send messages to a router if at least one consumer is available to receive them. Because AMQ Interconnect does not store messages, this credit-based flow control prevents producers from sending messages when there are no consumers present.

A client wishing to send a message to the router must wait until the router has provided it with credit. Attempting to publish a message without credit available will cause the client to block. Once credit is made available, the client will unblock, and the message will be sent to the router.

Note

Most AMQP client libraries enable you to determine the amount of credit available to a producer. For more information, consult your client’s documentation.

12.1.1.2. Addresses

Addresses determine how messages flow through your router network. An address designates an endpoint in your messaging network, such as:

  • Endpoint processes that consume data or offer a service
  • Topics that match multiple consumers to multiple producers

  • Entities within a messaging broker:

    • Queues
    • Durable Topics
    • Exchanges

When a router receives a message, it uses the message’s address to determine where to send the message (either its destination or one step closer to its destination).

AMQ Interconnect considers addresses to be mobile in that any user of an address may be directly connected to any router in the router network and may even move around the topology. In cases where messages are broadcast to or balanced across multiple consumers, the users of the address may be connected to multiple routers in the network.

Mobile addresses may be discovered during normal router operation or configured through management settings.

12.1.1.3. Routing patterns

Routing patterns define the paths that a message with a mobile address can take across a network. These routing patterns can be used for both direct routing, in which the router distributes messages between clients without a broker, and indirect routing, in which the router enables clients to exchange messages through a broker.

Routing patterns fall into two categories: Anycast (Balanced and Closest) and Multicast. There is no concept of "unicast" in which there is only one consumer for an address.

Anycast distribution delivers each message to one consumer whereas multicast distribution delivers each message to all consumers.

Each address has one of the following routing patterns, which define the path that a message with the address can take across the messaging network:

Balanced

An anycast method that allows multiple consumers to use the same address. Each message is delivered to a single consumer only, and AMQ Interconnect attempts to balance the traffic load across the router network.

If multiple consumers are attached to the same address, each router determines which outbound path should receive a message by considering each path’s current number of unsettled deliveries. This means that more messages will be delivered along paths where deliveries are settled at higher rates.

Note

AMQ Interconnect neither measures nor uses message settlement time to determine which outbound path to use.

In this scenario, the messages are spread across both receivers regardless of path length:

Figure 12.1. Balanced Message Routing

Balanced Message Routing
Closest

An anycast method in which every message is sent along the shortest path to reach the destination, even if there are other consumers for the same address.

AMQ Interconnect determines the shortest path based on the topology cost to reach each of the consumers. If there are multiple consumers with the same lowest cost, messages will be spread evenly among those consumers.

In this scenario, all messages sent by Sender will be delivered to Receiver 1:

Figure 12.2. Closest Message Routing

Closest Message Routing
Multicast

Messages are sent to all consumers attached to the address. Each consumer will receive one copy of the message.

In this scenario, all messages are sent to all receivers:

Figure 12.3. Multicast Message Routing

Multicast Message Routing

12.1.1.4. Message settlement and reliability

AMQ Interconnect can deliver messages with the following degrees of reliability:

  • At most once
  • At least once
  • Exactly once

The level of reliability is negotiated between the producer and the router when the producer establishes a link to the router. To achieve the negotiated level of reliability, AMQ Interconnect treats all messages as either pre-settled or unsettled.

Pre-settled
Sometimes called fire and forget, the router settles the incoming and outgoing deliveries and propagates the settlement to the message’s destination. However, it does not guarantee delivery.
Unsettled

AMQ Interconnect propagates the settlement between the producer and consumer. For an anycast address, the router associates the incoming delivery with the resulting outgoing delivery. Based on this association, the router propagates changes in delivery state from the consumer to the producer.

For a multicast address, the router associates the incoming delivery with all outbound deliveries. The router waits for each consumer to set their delivery’s final state. After all outgoing deliveries have reached their final state, the router sets a final delivery state for the original inbound delivery and passes it to the producer.

The following table describes the reliability guarantees for unsettled messages sent to an anycast or multicast address:

Final dispositionAnycastMulticast

accepted

The consumer accepted the message.

At least one consumer accepted the message, but no consumers rejected it.

released

The message did not reach its destination.

The message did not reach any of the consumers.

modified

The message may or may not have reached its destination. The delivery is considered to be "in-doubt" and should be re-sent if "at least once" delivery is required.

The message may or may not have reached any of the consumers. However, no consumers rejected or accepted it.

rejected

The consumer rejected the message.

At least one consumer rejected the message.

12.1.2. Configuring address semantics

You can route messages between clients without using a broker. In a brokerless scenario (sometimes called direct-routed messaging), AMQ Interconnect routes messages between clients directly.

To route messages between clients, you configure an address with a routing distribution pattern. When a router receives a message with this address, the message is routed to its destination or destinations based on the address’s routing distribution pattern.

Procedure

  1. In the /etc/qpid-dispatch/qdrouterd.conf configuration file, add an address section. 

    address {
    prefix: my_address
    distribution: multicast
    ...
}
    prefix | pattern

    The address or group of addresses to which the address settings should be applied. You can specify a prefix to match an exact address or beginning segment of an address. Alternatively, you can specify a pattern to match an address using wildcards.

    A prefix matches either an exact address or the beginning segment within an address that is delimited by either a . or / character. For example, the prefix my_address would match the address my_address as well as my_address.1 and my_address/1. However, it would not match my_address1.

    A pattern matches an address that corresponds to a pattern. A pattern is a sequence of words delimited by either a . or / character. You can use wildcard characters to represent a word. The * character matches exactly one word, and the # character matches any sequence of zero or more words.

    The * and # characters are reserved as wildcards. Therefore, you should not use them in the message address.

    For more information about creating address patterns, see Section 12.1.5, “Address pattern matching”.

    Note

    You can convert a prefix value to a pattern by appending /# to it. For example, the prefix a/b/c is equivalent to the pattern a/b/c/#.

    distribution

    The message distribution pattern. The default is balanced, but you can specify any of the following options:

    • balanced - Messages sent to the address will be routed to one of the receivers, and the routing network will attempt to balance the traffic load based on the rate of settlement.
    • closest - Messages sent to the address are sent on the shortest path to reach the destination. It means that if there are multiple receivers for the same address, only the closest one will receive the message.

    • multicast - Messages are sent to all receivers that are attached to the address in a publish/subscribe model.

      For more information about message distribution patterns, see Section 12.1.1.3, “Routing patterns”.

    For information about additional attributes, see address in the qdrouterd.conf man page.

  2. Add the same address section to any other routers that need to use the address.

    The address that you added to this router configuration file only controls how this router distributes messages sent to the address. If you have additional routers in your router network that should distribute messages for this address, then you must add the same address section to each of their configuration files.

12.1.3. Configuring addresses for prioritized message delivery

You can set the priority level of an address to control how AMQ Interconnect processes messages sent to that address. Within the scope of a connection, AMQ Interconnect attempts to process messages based on their priority. For a connection with a large volume of messages in flight, this lowers the latency for higher-priority messages.

Assigning a high priority level to an address does not guarantee that messages sent to the address will be delivered before messages sent to lower-priority addresses. However, higher-priority messages will travel more quickly through the router network than they otherwise would.

Note

You can also control the priority level of individual messages by setting the priority level in the message header. However, the address priority takes precedence: if you send a prioritized message to an address with a different priority level, the router will use the address priority level.

Procedure

  • In the /etc/qpid-dispatch/qdrouterd.conf configuration file, add or edit an address and assign a priority level.

    This example adds an address with the highest priority level. The router will attempt to deliver messages sent to this address before messages with lower priority levels. 

    address {
    prefix: my-high-priority-address
    priority: 9
    ...
}
    priority
    The priority level to assign to all messages sent to this address. The range of valid priority levels is 0-9, in which the higher the number, the higher the priority. The default is 4.

Additional resources

12.1.4. Configuring brokered messaging

If you require "store and forward" capabilities, you can configure AMQ Interconnect to use brokered messaging. In this scenario, clients connect to a router to send and receive messages, and the router routes the messages to or from queues on a message broker.

You can configure the following:

12.1.4.1. How AMQ Interconnect enables brokered messaging

Brokered messaging enables AMQ Interconnect to store messages on a broker queue. This requires a connection to the broker, a waypoint address to represent the broker queue, and autolinks to attach to the waypoint address.

An autolink is a link that is automatically created by the router to attach to a waypoint address. With autolinks, client traffic is handled on the router, not the broker. Clients attach their links to the router, and then the router uses internal autolinks to connect to the queue on the broker. Therefore, the queue will always have a single producer and a single consumer regardless of how many clients are attached to the router.

Using autolinks is a form of message routing, as distinct from link routing. It is recommended to use link routing if you want to use semantics associated with a consumer, for example, the undeliverable-here=true modified delivery state.

Figure 12.4. Brokered messaging

Brokered Messaging

In this diagram, the sender connects to the router and sends messages to my_queue. The router attaches an outgoing link to the broker, and then sends the messages to my_queue. Later, the receiver connects to the router and requests messages from my_queue. The router attaches an incoming link to the broker to receive the messages from my_queue, and then delivers them to the receiver.

You can also route messages to a sharded queue, which is a single, logical queue comprised of multiple, underlying physical queues. Using queue sharding, it is possible to distribute a single queue over multiple brokers. Clients can connect to any of the brokers that hold a shard to send and receive messages.

Figure 12.5. Brokered messaging with sharded queue

Brokered Messaging with Sharded Queue

In this diagram, a sharded queue (my_queue) is distributed across two brokers. The router is connected to the clients and to both brokers. The sender connects to the router and sends messages to my_queue. The router attaches an outgoing link to each broker, and then sends messages to each shard (by default, the routing distribution is balanced). Later, the receiver connects to the router and requests all of the messages from my_queue. The router attaches an incoming link to one of the brokers to receive the messages from my_queue, and then delivers them to the receiver.

12.1.4.2. Routing messages through broker queues

You can route messages to and from a broker queue to provide clients with access to the queue through a router. In this scenario, clients connect to a router to send and receive messages, and the router routes the messages to or from the broker queue.

You can route messages to a queue hosted on a single broker, or route messages to a sharded queue distributed across multiple brokers.

Procedure

  1. In the /etc/qpid-dispatch/qdrouterd.conf configuration file, add a waypoint address for the broker queue.

    A waypoint address identifies a queue on a broker to which you want to route messages. This example adds a waypoint address for the my_queue queue: 

    address {
    prefix: my_queue
    waypoint: yes
}
    prefix | pattern

    The address prefix or pattern that matches the broker queue to which you want to send messages. You can specify a prefix to match an exact address or beginning segment of an address. Alternatively, you can specify a pattern to match an address using wildcards.

    A prefix matches either an exact address or the beginning segment within an address that is delimited by either a . or / character. For example, the prefix my_address would match the address my_address as well as my_address.1 and my_address/1. However, it would not match my_address1.

    A pattern matches an address that corresponds to a pattern. A pattern is a sequence of words delimited by either a . or / character. You can use wildcard characters to represent a word. The * character matches exactly one word, and the # character matches any sequence of zero or more words.

    The * and # characters are reserved as wildcards. Therefore, you should not use them in the message address.

    For more information about creating address patterns, see Section 12.1.5, “Address pattern matching”.

    Note

    You can convert a prefix value to a pattern by appending /# to it. For example, the prefix a/b/c is equivalent to the pattern a/b/c/#.

    waypoint
    Set this attribute to yes so that the router handles messages sent to this address as a waypoint.

  2. Connect the router to the broker.

    1. Add an outgoing connection to the broker if one does not exist.

      If the queue is sharded across multiple brokers, you must add a connection for each broker. For more information, see Section 8.3, “Connecting to external AMQP containers”.

      Note

      If the connection to the broker fails, AMQ Interconnect automatically attempts to reestablish the connection and reroute message deliveries to any available alternate destinations. However, some deliveries could be returned to the sender with a RELEASED or MODIFIED disposition. Therefore, you should ensure that your clients can handle these deliveries appropriately (generally by resending them).

    2. If you want to send messages to the broker queue, add an outgoing autolink to the broker queue.

      If the queue is sharded across multiple brokers, you must add an outgoing autolink for each broker.

      This example configures an outgoing auto link to send messages to a broker queue: 

      autoLink {
    address: my_queue
    connection: my_broker
    direction: out
    ...
}
      address
      The address of the broker queue. When the autolink is created, it will be attached to this address.
      externalAddress
      An optional alternate address for the broker queue. You use an external address if the broker queue should have a different address than that which the sender uses. In this scenario, senders send messages to the address address, and then the router routes them to the broker queue represented by the externalAddress address.
      connection | containerID
      How the router should connect to the broker. You can specify either an outgoing connection (connection) or the container ID of the broker (containerID).
      direction
      Set this attribute to out to specify that this autolink can send messages from the router to the broker.

      For information about additional attributes, see autoLink in the qdrouterd.conf man page.

  3. If you want to receive messages from the broker queue, add an incoming autolink from the broker queue:

    If the queue is sharded across multiple brokers, you must add an outgoing autolink for each broker.

    This example configures an incoming auto link to receive messages from a broker queue: 

    autoLink {
    address: my_queue
    connection: my_broker
    direction: in
    ...
}
    address
    The address of the broker queue. When the autolink is created, it will be attached to this address.
    externalAddress
    An optional alternate address for the broker queue. You use an external address if the broker queue should have a different address than that which the receiver uses. In this scenario, receivers receive messages from the address address, and the router retrieves them from the broker queue represented by the externalAddress address.
    connection | containerID
    How the router should connect to the broker. You can specify either an outgoing connection (connection) or the container ID of the broker (containerID).
    direction
    Set this attribute to in to specify that this autolink can receive messages from the broker to the router.

    For information about additional attributes, see autoLink in the qdrouterd.conf man page.

12.1.4.3. Handling undeliverable messages

You handle undeliverable messages for an address by configuring autolinks that point to fallback destinations. A fallback destination (such as a queue on a broker) stores messages that are not directly routable to any consumers.

During normal message delivery, AMQ Interconnect delivers messages to the consumers that are attached to the router network. However, if no consumers are reachable, the messages are diverted to any fallback destinations that were configured for the address (if the autolinks that point to the fallback destinations are active). When a consumer reconnects and becomes reachable again, it receives the messages stored at the fallback destination.

Note

AMQ Interconnect preserves the original delivery order for messages stored at a fallback destination. However, when a consumer reconnects, any new messages produced while the queue is draining will be interleaved with the messages stored at the fallback destination.

Prerequisites

Procedure

This procedure enables fallback for an address and configures autolinks to connect to the broker queue that provides the fallback destination for the address.

  1. In the /etc/qpid-dispatch/qdrouterd.conf configuration file, enable fallback destinations for the address. 

    address {
    prefix: my_address
    enableFallback: yes
}

  2. Add an outgoing autolink to a queue on the broker.

    For the address for which you enabled fallback, if messages are not routable to any consumers, the router will use this autolink to send the messages to a queue on the broker. 

    autoLink {
    address: my_address.2
    direction: out
    connection: my_broker
    fallback: yes
}

  3. If you want the router to send queued messages to attached consumers as soon as they connect to the router network, add an incoming autolink.

    As soon as a consumer attaches to the router, it will receive the messages stored in the broker queue, along with any new messages sent by the producer. The original delivery order of the queued messages is preserved; however, the queued messages will be interleaved with the new messages.

    If you do not add the incoming autolink, the messages will be stored on the broker, but will not be sent to consumers when they attach to the router. 

    autoLink {
    address: my_address.2
    direction: in
    connection: my_broker
    fallback: yes
}

12.1.5. Address pattern matching

In some router configuration scenarios, you might need to use pattern matching to match a range of addresses rather than a single, literal address. Address patterns match any address that corresponds to the pattern.

An address pattern is a sequence of tokens (typically words) that are delimited by either . or / characters. They also can contain special wildcard characters that represent words:

  • * represents exactly one word
  • # represents zero or more words

Example 12.1. Address pattern

This address contains two tokens, separated by the / delimiter:

my/address

Example 12.2. Address pattern with wildcard

This address contains three tokens. The * is a wildcard, representing any single word that might be between my and address:

my/*/address

The following table shows some address patterns and examples of the addresses that would match them:

This pattern…​Matches…​But not…​

news/*

news/europe

news/usa

news

news/usa/sports

news/#

news

news/europe

news/usa/sports

europe

usa

news/europe/#

news/europe

news/europe/sports

news/europe/politics/fr

news/usa

europe

news/*/sports

news/europe/sports

news/usa/sports

news

news/europe/fr/sports