AMQ Interconnect supports the following industry-recognized standards and network protocols:

AMQ Interconnect is supported on Red Hat Enterprise Linux 6 and 7. See Red Hat AMQ 7 Supported Configurations for more information.

This section introduces some key concepts about AMQ Interconnect

In a network built of multiple, interconnected routers 'routing'
determines which connection to use to send a message directly
to its destination or one step closer to its destination.

In this document, sudo is used for any command that requires root privileges. You should always exercise caution when using sudo, as any changes can affect the entire system.

For more information about using sudo, see The sudo Command.

This example demonstrates how the router can connect clients by receiving and sending messages between them. It uses the router’s default configuration file and does not require a broker.

Figure 4.1. Peer-to-peer Communication

As the diagram indicates, the configuration consists of an AMQ Interconnect component with two clients connected to it: a sender and a receiver. The receiver wants to receive messages on a specific address, and the sender sends messages to that address.

A broker is not used in this example, so there is no "store and forward" mechanism in the middle. Instead, the messages flow from sender to receiver only if the receiver is online, and the sender can confirm that the messages have arrived at their destination.

This example uses a AMQ Python client to start a receiver client, and then send five messages from the sender client.

The router’s configuration is defined in the router configuration file. You can access this file to view and modify that configuration.

Before you can make changes to a router configuration file, you should understand how the file is structured.

The configuration file contains sections. A section is a configurable entity, and it contains a set of attribute name-value pairs that define the settings for that entity. The syntax is as follows:

sectionName {
    attributeName: attributeValue
    attributeName: attributeValue
    ...
}

You can use different methods for changing a router’s configuration based on whether the router is currently running, and whether you want the change to take effect immediately.

The router’s configuration file controls the way in which the router functions. The default configuration file contains the minimum number of settings required for the router to run. As you become more familiar with the router, you can add to or change these settings, or create your own configuration files.

When you installed AMQ Interconnect, the default configuration file was added at the following path: /etc/qpid-dispatch/qdrouterd.conf. It includes some basic configuration settings that define the router’s operating mode, how it listens for incoming connections, and routing patterns for the message routing mechanism.

router {
    mode: standalone 1
    id: Router.A 2
}

listener { 3
    host: 0.0.0.0 4
    port: amqp 5
    authenticatePeer: no 6
}

address { 7
    prefix: closest
    distribution: closest
}

address {
    prefix: multicast
    distribution: multicast
}

address {
    prefix: unicast
    distribution: closest
}

address {
    prefix: exclusive
    distribution: closest
}

address {
    prefix: broadcast
    distribution: multicast
}

The router’s default configuration settings enable the router to run with minimal configuration. However, you may need to change some of these settings for the router to run properly in your environment.

Listening for incoming connections involves setting the host and port on which the router should listen for traffic.

Configuring outgoing connections involves setting the host and port on which the router connects to other routers and brokers.

When a router connects to a broker, the broker might provide backup connection data that the router can use if the primary connection fails. If the primary connection fails, the router attempts to reconnect by using a combination of the primary and — if provided — backup connections in round-robin fashion until the connection is successful. For more information about viewing the backup connection data provided by the broker, see Section 10.3.2.2, “Managing Connectors”.

You can configure AMQ Interconnect to communicate with clients, routers, and brokers in a secure way by authenticating and encrypting the router’s connections. AMQ Interconnect supports the following security protocols:

Before you can secure incoming and outgoing connections using SSL/TLS encryption and authentication, you must first set up the SSL/TLS profile in the router’s configuration file.

If you plan to use SASL to authenticate connections, you must first add the SASL attributes to the router entity in the router’s configuration file. These attributes define a set of SASL parameters that can be used by the router’s incoming and outgoing connections.

You can secure incoming connections by configuring each connection’s listener entity for encryption, authentication, or both.

You can secure outgoing connections by configuring each connection’s connector entity for encryption, authentication, or both.

By using the GSSAPI SASL mechanism, you can configure AMQ Interconnect to authenticate incoming connections using Kerberos.

You can restrict the number of user connections, and control access to AMQP messaging resources by configuring policies.

[
    ["vhost", {
        "hostname": "example.com",  1
        "maxConnectionsPerUser": 10,  2
        "allowUnknownUser": true,  3
        "groups": {
            "admin": {
                "users": ["admin1", "admin2"],  4
                "remoteHosts": ["127.0.0.1", "::1"],  5
                "sources": "*",  6
                "targets": "*"  7
            },
            "$default": {
                "remoteHosts": "*",  8
                "sources": ["news*", "sports*", "chat*"],  9
                "targets": "chat*"  10
            }
        }
    }]
]

[
    ["vhost", {
        "hostname": "traders.com",  1
        "groups": {
            "traders": {
                "users": ["trader1", "trader2"],  2
                "maxFrameSize": 10000,
                "maxSessionWindow": 5000000,  3
                "maxSessions": 1  4
            },
            "feeds": {
                "users": ["nyse-feed", "nasdaq-feed"],  5
                "maxFrameSize": 60000,
                "maxSessionWindow": 1200000000,  6
                "maxSessions": 3  7
            }
        }
    }]
]

While you can use either message routing or link routing to deliver messages to a destination, they differ in several important ways. Understanding these differences will enable you to choose the proper routing approach for any particular use case.

With message routing, routing is performed on messages as producers send them to a router. When a message arrives on a router, the router routes the message and its settlement based on the message’s address and routing pattern.

With message routing, you can do the following:

8.2.5. Routing Messages Through a Broker Queue

You can route messages to and from a broker queue to provide clients with access to the queue through a router. In this scenario, clients connect to a router to send and receive messages, and the router routes the messages to or from the broker queue.

You can route messages to a queue hosted on a single broker, or route messages to a sharded queue distributed across multiple brokers.

Figure 8.6. Brokered Messaging

In this diagram, the sender connects to the router and sends messages to my_queue. The router attaches an outgoing link to the broker, and then sends the messages to my_queue. Later, the receiver connects to the router and requests messages from my_queue. The router attaches an incoming link to the broker to receive the messages from my_queue, and then delivers them to the receiver.

You can also route messages to a sharded queue, which is a single, logical queue comprised of multiple, underlying physical queues. Using queue sharding, it is possible to distribute a single queue over multiple brokers. Clients can connect to any of the brokers that hold a shard to send and receive messages.

Figure 8.7. Brokered Messaging with Sharded Queue

In this diagram, a sharded queue (my_queue) is distributed across two brokers. The router is connected to the clients and to both brokers. The sender connects to the router and sends messages to my_queue. The router attaches an outgoing link to each broker, and then sends messages to each shard (by default, the routing distribution is balanced). Later, the receiver connects to the router and requests all of the messages from my_queue. The router attaches an incoming link to one of the brokers to receive the messages from my_queue, and then delivers them to the receiver.

Procedure

1. Add a waypoint address.

   This address identifies the queue to which you want to route messages.

2. Add autolinks to connect the router to the broker.

   Autolinks connect the router to the broker queue identified by the waypoint address.

3. If the queue is sharded, add autolinks for each additional broker that hosts a shard.

8.2.5.1. Configuring Waypoint Addresses

A waypoint address identifies a queue on a broker to which you want to route messages. You need to configure the waypoint address on each router that needs to use the address. For example, if a client is connected to Router A to send messages to the broker queue, and another client is connected to Router B to receive those messages, then you would need to configure the waypoint address on both Router A and Router B.

Prerequisites

An incoming connection (listener) to which the clients can connect should be configured. This connection defines how the producers and consumers connect to the router to send and receive messages. For more information, see Adding Incoming Connections.

Procedure

• Create waypoint addresses on each router that needs to use the address:

  address {
      prefix: ADDRESS_PREFIX
      waypoint: yes
  }

  prefix | pattern

  The address prefix or pattern that matches the broker queue to which you want to send messages. You can specify a prefix to match an exact address or beginning segment of an address. Alternatively, you can specify a pattern to match an address using wildcards.

  A prefix matches either an exact address or the beginning segment within an address that is delimited by either a . or / character. For example, the prefix my_address would match the address my_address as well as my_address.1 and my_address/1. However, it would not match my_address1.

  A pattern matches an address that corresponds to a pattern. A pattern is a sequence of words delimited by either a . or / character. You can use wildcard characters to represent a word. The * character matches exactly one word, and the # character matches any sequence of zero or more words.

  The * and # characters are reserved as wildcards. Therefore, you should not use them in the message address.

  The following table shows some examples of address patterns:

  This pattern…​	Matches…​
  news	news
  news/*/sports	news/europe/sports and news/usa/sports, but not news or news/europe/fr/sports
  news/#	news, news/europe, news/usa, news/usa/sports

  Note

  You can convert a prefix value to a pattern by appending /# to it. For example, the prefix a/b/c is equivalent to the pattern a/b/c/#.

  waypoint

  Set this attribute to yes so that the router handles messages sent to this address as a waypoint.

8.2.5.2. Connecting a Router to the Broker

After you add waypoint addresses to identify the broker queue, you must connect a router to the broker using autolinks.

With autolinks, client traffic is handled on the router, not the broker. Clients attach their links to the router, and then the router uses internal autolinks to connect to the queue on the broker. Therefore, the queue will always have a single producer and a single consumer regardless of how many clients are attached to the router.

1. If this router is different than the router that is connected to the clients, then add the waypoint address.

2. Add an outgoing connection to the broker:

   connector {
       name: NAME
       host: HOST_NAME/ADDRESS
       port: PORT_NUMBER/NAME
       role: route-container
       ...
   }

   name

   The name of the connector. Specify a name that describes the broker.

   host

   Either an IP address (IPv4 or IPv6) or hostname on which the router should connect to the broker.

   port

   The port number or symbolic service name on which the router should connect to the broker.

   role

   Specify route-container to indicate that this connection is for an external container (broker).

   For information about additional attributes, see connector in the qdrouterd.conf man page.

3. If you want to send messages to the broker queue, create an outgoing autolink to the broker queue:

   autoLink {
       addr: ADDRESS
       connection: CONNECTOR_NAME
       dir: out
       ...
   }

   addr

   The address of the broker queue. When the autolink is created, it will be attached to this address.

   externalAddr

   An optional alternate address for the broker queue. You use an external address if the broker queue should have a different address than that which the sender uses. In this scenario, senders send messages to the addr address, and then the router routes them to the broker queue represented by the externalAddr address.

   connection | containerID

   How the router should connect to the broker. You can specify either an outgoing connection (connection) or the container ID of the broker (containerID).

   dir

   Set this attribute to out to specify that this autolink can send messages from the router to the broker.

   For information about additional attributes, see autoLink in the qdrouterd.conf man page.

4. If you want to receive messages from the broker queue, create an incoming autolink from the broker queue:

   autoLink {
       addr: ADDRESS
       connection: CONNECTOR_NAME
       dir: in
       ...
   }

   addr

   The address of the broker queue. When the autolink is created, it will be attached to this address.

   externalAddr

   An optional alternate address for the broker queue. You use an external address if the broker queue should have a different address than that which the receiver uses. In this scenario, receivers receive messages from the addr address, and the router retrieves them from the broker queue represented by the externalAddr address.

   connection | containerID

   How the router should connect to the broker. You can specify either an outgoing connection (connection) or the container ID of the broker (containerID).

   dir

   Set this attribute to in to specify that this autolink can receive messages from the broker to the router.

   For information about additional attributes, see autoLink in the qdrouterd.conf man page.

connector {  1
    name: broker
    role: route-container
    host: 198.51.100.1
    port: 61617
    saslMechanisms: ANONYMOUS
}

address {  2
    prefix: queue
    waypoint: yes
}

autoLink {  3
    addr: queue.first
    dir: in
    connection: broker
}

autoLink {  4
    addr: queue.first
    dir: out
    connection: broker
}

autoLink {  5
    addr: queue.second
    dir: in
    connection: broker
}

autoLink {  6
    addr: queue.second
    dir: out
    connection: broker
}

$ qdstat --autolinks
AutoLinks
  addr          dir  phs  extAddr  link  status    lastErr
  ========================================================
  queue.first   in   1                   inactive
  queue.first   out  0                   inactive
  queue.second  in   1                   inactive
  queue.second  out  0                   inactive

$ qdstat --autolinks
AutoLinks
  addr          dir  phs  extAddr  link  status  lastErr
  ===========================================================================
  queue.first   in   1             6     active
  queue.first   out  0             7     active
  queue.second  in   1                   failed  Node not found: queue.second
  queue.second  out  0                   failed  Node not found: queue.second

$ python simple_send.py -a 127.0.0.1/queue.first -m3
all messages confirmed

$ qdstat -a
Router Addresses
  class   addr           phs  distrib   in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  ========================================================================================================
  mobile  queue.first    1    balanced  0        0      0       0      0   0    0     0        0
  mobile  queue.first    0    balanced  0        1      0       0      3   3    0     0        0

$ python simple_recv.py -a 127.0.0.1:5672/queue.first -m3
{u'sequence': int32(1)}
{u'sequence': int32(2)}
{u'sequence': int32(3)}

$ qdstat -a
Router Addresses
  class   addr           phs  distrib   in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  ========================================================================================================
  mobile  queue.first    1    balanced  0        0      0       0      3   3    0     0        0
  mobile  queue.first    0    balanced  0        1      0       0      3   3    0     0        0

Link routing provides an alternative strategy for brokered messaging. A link route represents a private messaging path between a sender and a receiver in which the router passes the messages between end points. You can think of a link route as a "virtual connection" or "tunnel" that travels from a sender, through the router network, to a receiver.

With link routing, routing is performed on link-attach frames, which are chained together to form a virtual messaging path that directly connects a sender and receiver. Once a link route is established, the transfer of message deliveries, flow frames, and dispositions is performed across the link route.

                        Public Network
                       +-----------------+
                       |      +-----+    |
                       | B1   | Rp  |    |
                       |      +/--\-+    |
                       |      /    \     |
                       |     /      \    |
                       +----/--------\---+
                           /          \
                          /            \
                         /              \
         Private Net A  /                \ Private Net B
        +--------------/--+           +---\-------------+
        |         +---/-+ |           | +--\--+         |
        |  B2     | Ra  | |           | | Rb  |   C1    |
        |         +-----+ |           | +-----+         |
        |                 |           |                 |
        |                 |           |                 |
        +-----------------+           +-----------------+

connector {  1
    name: broker
    role: route-container
    host: 198.51.100.1
    port: 61617
    saslMechanisms: ANONYMOUS
}

linkRoute {  2
    prefix: b2
    dir: in
    connection: broker
}

linkRoute {  3
    prefix: b2
    dir: out
    connection: broker
}

AMQ Interconnect logs are broken into different categories called logging modules. Each module provides important information about a particular aspect of AMQ Interconnect.

Tue Jun  7 13:28:27 2016 ROUTER (trace) Node Router.C next hop set: Router.B
Tue Jun  7 13:28:27 2016 ROUTER (trace) Node Router.C valid origins: []
Tue Jun  7 13:28:27 2016 ROUTER (trace) Node Router.C cost: 2
Tue Jun  7 13:28:27 2016 ROUTER (trace) Node Router.B valid origins: []
Tue Jun  7 13:28:27 2016 ROUTER (trace) Node Router.B cost: 1

Tue Jun  7 13:28:25 2016 ROUTER (trace) Node Router.C cost: 1
Tue Jun  7 13:28:26 2016 ROUTER (trace) Node Router.A created: maskbit=2
Tue Jun  7 13:28:26 2016 ROUTER (trace) Node Router.A link set: link_id=1
Tue Jun  7 13:28:26 2016 ROUTER (trace) Node Router.A valid origins: ['Router.C']
Tue Jun  7 13:28:26 2016 ROUTER (trace) Node Router.A cost: 1
Tue Jun  7 13:28:27 2016 ROUTER (trace) Node Router.C valid origins: ['Router.A']

Tue Jun  7 13:42:07 2016 ROUTER_CORE (trace) Core action 'link_flow'
Tue Jun  7 13:42:08 2016 ROUTER_CORE (trace) Core action 'link_deliver'
Tue Jun  7 13:42:08 2016 ROUTER_CORE (trace) Core action 'send_to'
Tue Jun  7 13:42:08 2016 ROUTER_CORE (trace) Core action 'link_flow'

Tue Jun  7 13:50:21 2016 ROUTER_HELLO (trace) RCVD: HELLO(id=Router.B area=0 inst=1465307413 seen=['Router.A', 'Router.C']) 1
Tue Jun  7 13:50:21 2016 ROUTER_HELLO (trace) SENT: HELLO(id=Router.A area=0 inst=1465307416 seen=['Router.B']) 2
Tue Jun  7 13:50:22 2016 ROUTER_HELLO (trace) RCVD: HELLO(id=Router.B area=0 inst=1465307413 seen=['Router.A', 'Router.C'])
Tue Jun  7 13:50:22 2016 ROUTER_HELLO (trace) SENT: HELLO(id=Router.A area=0 inst=1465307416 seen=['Router.B'])

Tue Jun  7 13:50:18 2016 ROUTER_HELLO (trace) SENT: HELLO(id=Router.B area=0 inst=1465307413 seen=['Router.A', 'Router.C']) 1
Tue Jun  7 13:50:18 2016 ROUTER_HELLO (trace) RCVD: HELLO(id=Router.A area=0 inst=1465307416 seen=['Router.B']) 2
Tue Jun  7 13:50:19 2016 ROUTER_HELLO (trace) RCVD: HELLO(id=Router.C area=0 inst=1465307411 seen=['Router.B']) 3

Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) SENT: LSR(id=Router.A area=0) to: Router.C //
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) SENT: LSR(id=Router.A area=0) to: Router.B //
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) SENT: RA(id=Router.A area=0 inst=1465308600 ls_seq=1 mobile_seq=1) 1
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) RCVD: LSU(id=Router.B area=0 inst=1465308595 ls_seq=2 ls=LS(id=Router.B area=0 ls_seq=2 peers={'Router.A': 1L, 'Router.C': 1L})) 2
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) RCVD: LSR(id=Router.B area=0)
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) SENT: LSU(id=Router.A area=0 inst=1465308600 ls_seq=1 ls=LS(id=Router.A area=0 ls_seq=1 peers={'Router.B': 1}))
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) RCVD: RA(id=Router.C area=0 inst=1465308592 ls_seq=1 mobile_seq=0)
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) SENT: LSR(id=Router.A area=0) to: Router.C
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) RCVD: LSR(id=Router.C area=0) 3
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) SENT: LSU(id=Router.A area=0 // inst=1465308600 ls_seq=1 ls=LS(id=Router.A area=0 ls_seq=1 peers={'Router.B': 1}))
Tue Jun  7 14:10:02 2016 ROUTER_LS (trace) RCVD: LSU(id=Router.C area=0 inst=1465308592 ls_seq=1 ls=LS(id=Router.C area=0 ls_seq=1 peers={'Router.B': 1L})) 4
Tue Jun  7 14:10:03 2016 ROUTER_LS (trace) Computed next hops: {'Router.C': 'Router.B', 'Router.B': 'Router.B'} 5
Tue Jun  7 14:10:03 2016 ROUTER_LS (trace) Computed costs: {'Router.C': 2L, 'Router.B': 1}
Tue Jun  7 14:10:03 2016 ROUTER_LS (trace) Computed valid origins: {'Router.C': [], 'Router.B': []}

Tue Jun  7 14:27:20 2016 ROUTER_MA (trace) SENT: MAU(id=Router.A area=0 mobile_seq=1 add=['Cmy_queue', 'Dmy_queue', 'M0my_queue_wp'] del=[]) 1
Tue Jun  7 14:27:21 2016 ROUTER_MA (trace) RCVD: MAR(id=Router.C area=0 have_seq=0) 2
Tue Jun  7 14:27:21 2016 ROUTER_MA (trace) SENT: MAU(id=Router.A area=0 mobile_seq=1 add=['Cmy_queue', 'Dmy_queue', 'M0my_queue_wp'] del=[])
Tue Jun  7 14:27:22 2016 ROUTER_MA (trace) RCVD: MAR(id=Router.B area=0 have_seq=0) 3
Tue Jun  7 14:27:22 2016 ROUTER_MA (trace) SENT: MAU(id=Router.A area=0 mobile_seq=1 add=['Cmy_queue', 'Dmy_queue', 'M0my_queue_wp'] del=[])
Tue Jun  7 14:27:39 2016 ROUTER_MA (trace) RCVD: MAU(id=Router.C area=0 mobile_seq=1 add=['M0my_test'] del=[]) 4
Tue Jun  7 14:27:51 2016 ROUTER_MA (trace) RCVD: MAU(id=Router.C area=0 mobile_seq=2 add=[] del=['M0my_test']) 5

Tue Jun  7 14:36:54 2016 MESSAGE (trace) Sending Message{to='amqp:/_topo/0/Router.B/qdrouter' body='\d1\00\00\00\1b\00\00\00\04\a1\02id\a1\08R'} on link qdlink.p9XmBm19uDqx50R
Tue Jun  7 14:36:54 2016 MESSAGE (trace) Received Message{to='amqp:/_topo/0/Router.A/qdrouter' body='\d1\00\00\00\8e\00\00\00
\a1\06ls_se'} on link qdlink.phMsJOq7YaFsGAG
Tue Jun  7 14:36:54 2016 MESSAGE (trace) Received Message{ body='\d1\00\00\00\10\00\00\00\02\a1\08seque'} on link qdlink.FYHqBX+TtwXZHfV
Tue Jun  7 14:36:54 2016 MESSAGE (trace) Sending Message{ body='\d1\00\00\00\10\00\00\00\02\a1\08seque'} on link qdlink.yU1tnPs5KbMlieM
Tue Jun  7 14:36:54 2016 MESSAGE (trace) Sending Message{to='amqp:/_local/qdhello' body='\d1\00\00\00G\00\00\00\08\a1\04seen\d0'} on link qdlink.p9XmBm19uDqx50R
Tue Jun  7 14:36:54 2016 MESSAGE (trace) Sending Message{to='amqp:/_topo/0/Router.C/qdrouter' body='\d1\00\00\00\1b\00\00\00\04\a1\02id\a1\08R'} on link qdlink.p9XmBm19uDqx50R

Tue Jun  7 14:39:52 2016 SERVER (trace) [2]:  <- AMQP
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:  <- AMQP
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:0 <- @open(16) [container-id="Router.B", max-frame-size=16384, channel-max=32767, idle-time-out=8000, offered-capabilities=:"ANONYMOUS-RELAY", properties={:product="qpid-dispatch-router", :version="0.6.0"}]
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:0 -> @begin(17) [next-outgoing-id=0, incoming-window=15, outgoing-window=2147483647]
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:RAW: "\x00\x00\x00\x1e\x02\x00\x00\x00\x00S\x11\xd0\x00\x00\x00\x0e\x00\x00\x00\x04@R\x00R\x0fp\x7f\xff\xff\xff"
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:1 -> @begin(17) [next-outgoing-id=0, incoming-window=15, outgoing-window=2147483647]
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:RAW: "\x00\x00\x00\x1e\x02\x00\x00\x01\x00S\x11\xd0\x00\x00\x00\x0e\x00\x00\x00\x04@R\x00R\x0fp\x7f\xff\xff\xff"
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:0 -> @attach(18) [name="qdlink.uSSeXPSfTHhxo8d", handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, expiry-policy=:"link-detach", timeout=0, dynamic=false, capabilities=:"qd.router"], target=@target(41) [durable=0, expiry-policy=:"link-detach", timeout=0, dynamic=false, capabilities=:"qd.router"], initial-delivery-count=0]
Tue Jun  7 14:39:52 2016 SERVER (trace) [1]:RAW: "\x00\x00\x00\x91\x02\x00\x00\x00\x00S\x12\xd0\x00\x00\x00\x81\x00\x00\x00\x0a\xa1\x16qdlink.uSSeXPSfTHhxo8dR\x00AP\x02P\x00\x00S(\xd0\x00\x00\x00'\x00\x00\x00\x0b@R\x00\xa3\x0blink-detachR\x00B@@@@@\xa3\x09qd.router\x00S)\xd0\x00\x00\x00#\x00\x00\x00\x07@R\x00\xa3\x0blink-detachR\x00B@\xa3\x09qd.router@@R\x00"

Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: ConnectorEntity(addr=127.0.0.1, allowRedirect=True, cost=1, host=127.0.0.1, identity=connector/127.0.0.1:5672:BROKER, idleTimeoutSeconds=16, maxFrameSize=65536, name=BROKER, port=5672, role=route-container, stripAnnotations=both, type=org.apache.qpid.dispatch.connector, verifyHostName=True)
Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: RouterConfigAddressEntity(distribution=closest, identity=router.config.address/0, name=router.config.address/0, prefix=my_address, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: RouterConfigAddressEntity(distribution=balanced, identity=router.config.address/1, name=router.config.address/1, prefix=my_queue_wp, type=org.apache.qpid.dispatch.router.config.address, waypoint=True)
Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: RouterConfigLinkrouteEntity(connection=BROKER, dir=in, distribution=linkBalanced, identity=router.config.linkRoute/0, name=router.config.linkRoute/0, prefix=my_queue, type=org.apache.qpid.dispatch.router.config.linkRoute)
Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: RouterConfigLinkrouteEntity(connection=BROKER, dir=out, distribution=linkBalanced, identity=router.config.linkRoute/1, name=router.config.linkRoute/1, prefix=my_queue, type=org.apache.qpid.dispatch.router.config.linkRoute)
Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: RouterConfigAutolinkEntity(addr=my_queue_wp, connection=BROKER, dir=in, identity=router.config.autoLink/0, name=router.config.autoLink/0, type=org.apache.qpid.dispatch.router.config.autoLink)
Tue Jun  7 15:07:32 2016 AGENT (debug) Add entity: RouterConfigAutolinkEntity(addr=my_queue_wp, connection=BROKER, dir=out, identity=router.config.autoLink/1, name=router.config.autoLink/1, type=org.apache.qpid.dispatch.router.config.autoLink)

Tue Jun  7 14:46:18 2016 CONTAINER (trace) Container Initialized
Tue Jun  7 14:46:18 2016 CONTAINER (trace) Node Type Registered - router
Tue Jun  7 14:46:18 2016 CONTAINER (trace) Node of type 'router' installed as default node

$ sudo qdrouterd --conf xxx
Wed Jun 15 09:53:28 2016 ERROR (error) Python: Exception: Cannot load configuration file xxx: [Errno 2] No such file or directory: 'xxx'
Wed Jun 15 09:53:28 2016 ERROR (error) Traceback (most recent call last):
  File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/management/config.py", line 155, in configure_dispatch
    config = Config(filename)
  File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/management/config.py", line 41, in __init__
    self.load(filename, raw_json)
  File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/management/config.py", line 123, in load
    with open(source) as f:
Exception: Cannot load configuration file xxx: [Errno 2] No such file or directory: 'xxx'

Wed Jun 15 09:53:28 2016 MAIN (critical) Router start-up failed: Python: Exception: Cannot load configuration file xxx: [Errno 2] No such file or directory: 'xxx'
qdrouterd: Python: Exception: Cannot load configuration file xxx: [Errno 2] No such file or directory: 'xxx'

Tue Jun  7 15:07:32 2016 POLICY (info) Policy configured maximumConnections: 0, policyFolder: '', access rules enabled: 'false'
Tue Jun  7 15:07:32 2016 POLICY (info) Policy fallback defaultApplication is disabled

You can specify the types of events that should be logged, the format of the log entries, and where those entries should be sent.

You may need to view log entries to diagnose errors, performance problems, and other important issues. A log entry consists of an optional timestamp, the logging module, the logging level, and the log message.

If you prefer to use a graphic interface to manage AMQ, you can use AMQ Console. AMQ Console is a web console included in the AMQ Broker installation, and it enables you to use a web browser to manage AMQ Broker and AMQ Interconnect.

For more information, see Using AMQ Console.

You can use qdstat to view the status of routers on your router network. For example, you can view information about the attached links and configured addresses, available connections, and nodes in the router network.

$ qdstat OPTION [CONNECTION_OPTIONS] [SECURE_CONNECTION_OPTIONS]

You can use qdmanage to view and modify the configuration of a running router at runtime. Specifically, qdmanage enables you to create, read, update, and delete the sections and attributes in the router’s configuration file without having to restart the router.

$ qdmanage [CONNECTION_OPTIONS] OPERATION [OPTIONS]

$ qdmanage query --type listener
[
  {
    "stripAnnotations": "both",
    "addr": "127.0.0.1",
    "multiTenant": false,
    "requireSsl": false,
    "idleTimeoutSeconds": 16,
    "saslMechanisms": "ANONYMOUS",
    "maxFrameSize": 16384,
    "requireEncryption": false,
    "host": "0.0.0.0",
    "cost": 1,
    "role": "normal",
    "http": false,
    "maxSessions": 32768,
    "authenticatePeer": false,
    "type": "org.apache.qpid.dispatch.listener",
    "port": "amqp",
    "identity": "listener/0.0.0.0:amqp",
    "name": "listener/0.0.0.0:amqp"
  }
]

qdmanage query --type=listener

qdmanage query role port --type=listener

qdmanage read --name=LISTENER_NAME

qdmanage create --type=listener --ATTRIBUTE=VALUE ...

qdmanage update --type=listener --ATTRIBUTE=VALUE ...

qdmanage update --type=listener --ATTRIBUTE

qdmanage delete --name=LISTENER_NAME

qdmanage query --type=connector

qdmanage query role port --type=connector

qdmanage query failoverList --type=connector --name=CONNECTOR_NAME

qdmanage read --name=CONNECTOR_NAME

qdmanage create --type=connector --ATTRIBUTE=VALUE ...

qdmanage update --type=connector --ATTRIBUTE=VALUE ...

qdmanage update --type=connector --ATTRIBUTE

qdmanage delete --name=CONNECTOR_NAME

qdmanage query --type=sslProfile

qdmanage create --type=sslProfile --name=NAME --certDB=PATH --certFile=PATH --keyFile=PATH --ATTRIBUTE=VALUE ...

qdmanage update --name=LISTENER_NAME --sslProfile=NAME --requireSsl=yes

qdmanage update --name=LISTENER_NAME --ATTRIBUTE=VALUE ...

qdmanage update --name=LISTENER_NAME --authenticatePeer=yes

qdmanage update --name=LISTENER_NAME --authenticatePeer=no

qdmanage update --name=CONNECTOR_NAME --sslProfile=NAME

qdmanage update --name=CONNECTOR_NAME --sslProfile

qdmanage delete --name=SSL_PROFILE_NAME

qdmanage update --type=router --saslConfigPath=PATH --saslConfigName=NAME

qdmanage update --name=LISTENER_NAME --authenticatePeer=yes --saslMechanisms=MECHANISMS

qdmanage update --name=LISTENER_NAME --saslMechanisms=MECHANISMS

qdmanage update --name=CONNECTOR_NAME --saslMechanisms=MECHANISMS --saslUsername=USERNAME --saslPassword=PASSWORD

qdmanage update --name=CONNECTOR_NAME --saslMechanisms=MECHANISMS

qdmanage update --name=LISTENER_NAME --requireEncryption=yes --saslMechanisms=MECHANISMS

qdmanage update --name=LISTENER_NAME --saslMechanisms=MECHANISMS

qdmanage update --name=LISTENER_NAME --requireEncryption=no --saslMechanisms

qdmanage update --type=router --saslConfigPath --saslConfigName

qdmanage query --type=address

qdmanage read --name=ADDRESS_NAME

qdmanage query prefix distribution --type=address

qdmanage query prefix --type=address --waypoint=yes

qdmanage query --type=autolink

qdmanage create --type=address --prefix=ADDRESS_PREFIX --distribution=DISTRIBUTION_PATTERN ...

qdmanage update --name=ADDRESS_NAME --ATTRIBUTE=VALUE ...

qdmanage update --name=AUTOLINK_NAME --ATTRIBUTE=VALUE ...

qdmanage delete --name=ADDRESS_NAME

qdmanage delete --name=AUTOLINK_NAME

qdmanage query --type=linkRoute

qdmanage read --name=LINK_ROUTE_NAME

qdmanage update --name=LINK_ROUTE_NAME --ATTRIBUTE=VALUE ...

qdmanage delete --name=INCOMING_LINK_ROUTE_NAME
qdmanage delete --name=OUTGOING_LINK_ROUTE_NAME

qdmanage query --type=log

qdmanage read --type=log --module=LOGGING_MODULE_NAME

qdmanage create --type=log --module=DEFAULT --enable=LOGGING_LEVEL --timestamp=yes --ATTRIBUTE=VALUE

qdmanage create --type=log --module=LOGGING_MODULE_NAME --enable=LOGGING_LEVEL --ATTRIBUTE=VALUE ...

qdmanage update --type=log --module=LOGGING_MODULE_NAME --ATTRIBUTE=VALUE ...

Offering path redundancy means designing the network topology in a way that even when one or more routers go down or even connections between them, each destination is always reachable following alternate paths through the routers that are still part of the network.

Consider the following simple scenario :

Figure 11.1. Path Redundancy Enabled Topology

The "Router.A" configuration is something like following.

router {
    mode: interior
    id: Router.A
}

listener {
    host: 0.0.0.0
    port: 6000
    authenticatePeer: no
}

connector {
    name: INTER_ROUTER_B
    addr: 127.0.0.1
    port: 5001
    role: inter-router
}

connector {
    name: INTER_ROUTER_C
    addr: 127.0.0.1
    port: 5002
    role: inter-router
}

There is only one listener in order to accept client connections and two connector entities for connecting to the other two routers.

The "Router.B" configuration is the following.

router {
    mode: interior
    id: Router.B
}

listener {
    addr: 0.0.0.0
    port: 5001
    authenticatePeer: no
    role: inter-router
}

listener {
    host: 0.0.0.0
    port: 6001
    authenticatePeer: no
}

It has two listener entities in order to listen for connections from clients and from other routers in the network (in this case from the "Router.A" and "Router.C").

Finally, quite similar is the "Router.C" configuration.

router {
    mode: interior
    id: Router.C
}

listener {
    addr: 0.0.0.0
    port: 5002
    authenticatePeer: no
    role: inter-router
}

listener {
    host: 0.0.0.0
    port: 6002
    authenticatePeer: no
}

connector {
    name: INTER_ROUTER_B
    addr: 127.0.0.1
    port: 5001
    role: inter-router
}

It has two listener entities in order to listen for connections from clients and from other routers in the network (in this case from the "Router.A") and finally it has a connector (for connecting to the "Router.B")

Consider a sender client connected to "Router.A" and attached to my_address address which start to send messages (that is, 10 messages) and a receiver client connected to the "Router.B" and attached to the same address.

Starting the receiver, it waits for messages with no output on the console.

$ sudo python simple_recv.py -a localhost:6001/my_queue -m 10

Starting the sender, all the messages flow through "Router.A" and "Router.B" reaching the receiver; at this point the messages are all confirmed at sender side.

$ sudo python simple_send.py -a localhost:6001/my_queue -m 10
all messages confirmed

At same time, the receivers shows the messages received through the "Router.B".

{u'sequence': 1L}
{u'sequence': 2L}
{u'sequence': 3L}
{u'sequence': 4L}
{u'sequence': 5L}
{u'sequence': 6L}
{u'sequence': 7L}
{u'sequence': 8L}
{u'sequence': 9L}
{u'sequence': 10L}

The path redundancy is provided by the other available path through the "Router.A", "Router.C" and then "Router.B". It means that if the connection between "Router.A" and "Router.B" goes down, the alternative path is used to reach the receiver.

Now, consider a fault on the "Router.B"; the receiver is not reachable anymore on that path but it can connect to the "Router.C" in order to continue to receive messages from the sender which does not know what’s happened and it can continue to send messages to the "Router.A" in order to reach the receiver.

Figure 11.2. Path Redundancy after Router Failure

The receiver is still reachable in order to get messages from the sender as displayed in the console output.

$ sudo python simple_recv.py -a localhost:6002/my_queue -m 10
{u'sequence': 1L}
{u'sequence': 2L}
{u'sequence': 3L}
{u'sequence': 4L}
{u'sequence': 5L}
{u'sequence': 6L}
{u'sequence': 7L}
{u'sequence': 8L}
{u'sequence': 9L}
{u'sequence': 10L}

In order to have temporal decoupling in a solution based on AMQ Interconnect, adding one or more brokers is a must for its "store and forward" feature. Choosing the right topology, it is possible to have a solution which offers reliability with both path redundancy and permanent storing for messages.

Consider the following simple scenario :

Figure 11.3. Path Redundancy and Temporal Decoupling Enabled Topology

The receiver client can be offline when the sender starts to send messages because they’ll be stored into the queue permanently; coming back online, the receiver can get messages from the queue itself without message loss.

The "Router.A" configuration is something like following.

router {
    mode: interior
    id: Router.A
}

listener {
    host: 0.0.0.0
    port: 6000
    authenticatePeer: no
}

connector {
    name: INTER_ROUTER_B
    addr: 127.0.0.1
    port: 5001
    role: inter-router
}

connector {
    name: INTER_ROUTER_C
    addr: 127.0.0.1
    port: 5002
    role: inter-router
}

address {
    prefix: my_queue
    waypoint: yes
}

It has a listener for accepting incoming connections from clients and two connector entities in order to connect to the other routers. The queue named my_queue on the broker is exposed by a waypoint.

The "Router.B" configuration is the following.

router {
    mode: interior
    id: Router.B
}

listener {
    addr: 0.0.0.0
    port: 5001
    authenticatePeer: no
    role: inter-router
}

listener {
    host: 0.0.0.0
    port: 6001
    authenticatePeer: no
}

connector {
    name: BROKER
    addr: 127.0.0.1
    port: 5672
    role: route-container
}

address {
    prefix: my_queue
    waypoint: yes
}

autoLink {
    addr: my_queue
    connection: BROKER
    dir: in
}

autoLink {
    addr: my_queue
    connection: BROKER
    dir: out
}

It can accept incoming connections from clients and from other routers (in this case the "Router.A") and connects to the broker. The queue named my_queue on the broker is exposed by a waypoint with the related auto-links in both directions in order to send and receive messages to/from the queue itself.

Finally, the simple "Router.C" configuration.

router {
    mode: interior
    id: Router.C
}

listener {
    addr: 0.0.0.0
    port: 5002
    authenticatePeer: no
    role: inter-router
}

listener {
    host: 0.0.0.0
    port: 6002
    authenticatePeer: no
}

It can accept incoming connections from clients and from other routers (in this case the "Router.A"). Initially there is no connection between this router and the broker.

First of all, thanks to the broker and its "store and forward" feature, the sender can connect to the "Router.A" and start to send messages even if the receiver is not online in that moment. Using the Python sample from the Qpid Proton library, the console output is like following.

$ sudo python simple_send.py -a localhost:6000/my_queue -m 10
all messages confirmed

All messages are confirmed because they reached the queue inside the broker through "Router.A" and "Router.B"; it is confirmed using the qdstat tool.

$ sudo qdstat -b localhost:6001 -a
Router Addresses
  class   addr                   phs  distrib    in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  =================================================================================================================
  local   $_management_internal       closest    1        0      0       0      0   0    0     0        0
  local   $displayname                closest    1        0      0       0      0   0    0     0        0
  mobile  $management            0    closest    1        0      0       0      1   0    0     1        0
  local   $management                 closest    1        0      0       0      0   0    0     0        0
  router  Router.A                    closest    0        0      1       0      0   0    6     0        6
  router  Router.C                    closest    0        0      1       0      0   0    4     0        4
  mobile  my_queue               1    balanced   0        0      0       0      0   0    0     0        0
  mobile  my_queue               0    balanced   0        1      0       0      0   10   0     0        0
  local   qdhello                     flood      1        1      0       0      0   0    0     97       117
  local   qdrouter                    flood      1        0      0       0      0   0    0     7        0
  topo    qdrouter                    flood      1        0      2       0      0   0    8     13       9
  local   qdrouter.ma                 multicast  1        0      0       0      0   0    0     2        0
  topo    qdrouter.ma                 multicast  1        0      2       0      0   0    0     0        1
  local   temp.7f2u0zv9_U6QC5e        closest    0        1      0       0      0   0    0     0        0

For the "Router.B", there are 10 messages as output (from the router to the broker) on the my_queue address.

Starting the receiver connected to the "Router.A", it gets all the available messages from the queue.

$ sudo python simple_recv.py -a localhost:6000/my_queue -m 10
{u'sequence': 1L}
{u'sequence': 2L}
{u'sequence': 3L}
{u'sequence': 4L}
{u'sequence': 5L}
{u'sequence': 6L}
{u'sequence': 7L}
{u'sequence': 8L}
{u'sequence': 9L}
{u'sequence': 10L}

Using the qdstat tool on the "Router.B" another time, the output is like following.

$ sudo qdstat -b localhost:6001 -a
Router Addresses
  class   addr                   phs  distrib    in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  =================================================================================================================
  local   $_management_internal       closest    1        0      0       0      0   0    0     0        0
  local   $displayname                closest    1        0      0       0      0   0    0     0        0
  mobile  $management            0    closest    1        0      0       0      2   0    0     2        0
  local   $management                 closest    1        0      0       0      0   0    0     0        0
  router  Router.A                    closest    0        0      1       0      0   0    6     0        6
  router  Router.C                    closest    0        0      1       0      0   0    4     0        4
  mobile  my_queue               1    balanced   0        0      0       0      10  0    10    0        0
  mobile  my_queue               0    balanced   0        1      0       0      0   10   0     0        0
  local   qdhello                     flood      1        1      0       0      0   0    0     156      182
  local   qdrouter                    flood      1        0      0       0      0   0    0     7        0
  topo    qdrouter                    flood      1        0      2       0      0   0    10    18       11
  local   qdrouter.ma                 multicast  1        0      0       0      0   0    0     2        0
  topo    qdrouter.ma                 multicast  1        0      2       0      0   0    0     2        1
  local   temp.Xov_ZUcyti3jjXY        closest    0        1      0       0      0   0    0     0        0

For the "Router.B", there are 10 messages as input (from the broker to the router) on the my_queue address.

Now, consider a fault on the "Router.B"; in this case the broker is not reachable but it is possible to set up path redundancy through the "Router.C".

Figure 11.4. Path Redundancy and Temporal Decoupling after Router Failure

Using the qdmanage tool, it is possible to configure the waypoint on my_queue address, the related auto-links in both directions and finally the connector instance in order to enable the connection to the broker.

$ sudo qdmanage -b localhost:6002 create --stdin
[
{ "type":"connector", "name":"BROKER", "port":5672, "role":"route-container" },
{ "type":"address", "prefix":"my_queue", "waypoint":"yes" },
{ "type":"autoLink", "addr":"my_queue", "connection":"BROKER", "dir":"in" },
{ "type":"autoLink", "addr":"my_queue", "connection":"BROKER", "dir":"out" }
]
[
  {
    "verifyHostName": true,
    "stripAnnotations": "both",
    "name": "BROKER",
    "allowRedirect": true,
    "idleTimeoutSeconds": 16,
    "maxFrameSize": 65536,
    "host": "127.0.0.1",
    "cost": 1,
    "role": "route-container",
    "maxSessions": 32768,
    "type": "org.apache.qpid.dispatch.connector",
    "port": "5672",
    "identity": "connector/127.0.0.1:5672:BROKER",
    "addr": "127.0.0.1"
  },
  {
    "name": null,
    "prefix": "my_queue",
    "ingressPhase": 0,
    "waypoint": false,
    "distribution": "balanced",
    "type": "org.apache.qpid.dispatch.router.config.address",
    "identity": "7",
    "egressPhase": 0
  },
  {
    "addr": "my_queue",
    "name": null,
    "linkRef": null,
    "type": "org.apache.qpid.dispatch.router.config.autoLink",
    "operStatus": "inactive",
    "connection": "BROKER",
    "dir": "in",
    "phase": 1,
    "lastError": null,
    "externalAddr": null,
    "identity": "8",
    "containerId": null
  },
  {
    "addr": "my_queue",
    "name": null,
    "linkRef": null,
    "type": "org.apache.qpid.dispatch.router.config.autoLink",
    "operStatus": "inactive",
    "connection": "BROKER",
    "dir": "out",
    "phase": 0,
    "lastError": null,
    "externalAddr": null,
    "identity": "9",
    "containerId": null
  }
]

The "Router.C" configuration changes in the same way as "Router.B". It can accept incoming connections from clients and from other routers (in this case the "Router.A") and connects to the broker. The queue named my_queue on the broker is exposed by a waypoint with the related auto-links in both directions in order to send and receive messages to/from the queue itself.

At this point, the sender can connect to the "Router.A" for sending messages to the queue in the broker thanks to the "Router.C".

$ sudo python simple_send.py -a localhost:6000/my_queue -m 10
all messages confirmed

All messages are confirmed because they reached the queue inside the broker through "Router.A" and "Router.C"; it is confirmed using the qdstat tool.

$ sudo qdstat -b localhost:6002 -a
Router Addresses
  class   addr                   phs  distrib    in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  =================================================================================================================
  local   $_management_internal       closest    1        0      0       0      0   0    0     1        1
  local   $displayname                closest    1        0      0       0      0   0    0     0        0
  mobile  $management            0    closest    1        0      0       0      5   0    0     5        0
  local   $management                 closest    1        0      0       0      0   0    0     0        0
  router  Router.A                    closest    0        0      1       0      0   0    5     0        5
  mobile  my_queue               0    balanced   0        1      0       0      0   10   0     0        0
  mobile  my_queue               1    balanced   0        0      0       0      0   0    0     0        0
  local   qdhello                     flood      1        1      0       0      0   0    0     665      647
  local   qdrouter                    flood      1        0      0       0      0   0    0     8        0
  topo    qdrouter                    flood      1        0      1       0      0   0    31    52       32
  local   qdrouter.ma                 multicast  1        0      0       0      0   0    0     1        0
  topo    qdrouter.ma                 multicast  1        0      1       0      0   0    1     2        1
  local   temp.k6UMaS4P0JmtSlL        closest    0        1      0       0      0   0    0     0        0

For the "Router.C", there are 10 messages as output (from the router to the broker) on the my_queue address.

Starting the receiver connected to the "Router.A", it gets all the available messages from the queue.

$ sudo python simple_recv.py -a localhost:6000/my_queue -m 10
{u'sequence': 1L}
{u'sequence': 2L}
{u'sequence': 3L}
{u'sequence': 4L}
{u'sequence': 5L}
{u'sequence': 6L}
{u'sequence': 7L}
{u'sequence': 8L}
{u'sequence': 9L}
{u'sequence': 10L}

Using the qdstat tool on the "Router.C" another time, the output is like following.

$ sudo qdstat -b localhost:6002 -a
Router Addresses
  class   addr                   phs  distrib    in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  =================================================================================================================
  local   $_management_internal       closest    1        0      0       0      0   0    0     1        1
  local   $displayname                closest    1        0      0       0      0   0    0     0        0
  mobile  $management            0    closest    1        0      0       0      6   0    0     6        0
  local   $management                 closest    1        0      0       0      0   0    0     0        0
  router  Router.A                    closest    0        0      1       0      0   0    5     0        5
  mobile  my_queue               0    balanced   0        1      0       0      0   10   0     0        0
  mobile  my_queue               1    balanced   0        0      0       0      10  0    10    0        0
  local   qdhello                     flood      1        1      0       0      0   0    0     746      726
  local   qdrouter                    flood      1        0      0       0      0   0    0     8        0
  topo    qdrouter                    flood      1        0      1       0      0   0    34    55       35
  local   qdrouter.ma                 multicast  1        0      0       0      0   0    0     1        0
  topo    qdrouter.ma                 multicast  1        0      1       0      0   0    1     4        1
  local   temp.Hso3moy3l+Sn+Fy        closest    0        1      0       0      0   0    0     0        0

For the "Router.C", there are 10 messages as input (from the broker to the router) on the my_queue address.

Every broker has limits in terms of queue size but in order to overcome this problem, one possible solution is "sharding" queues : in that way a single queue is divided in more "shards" (chunks) each on a different broker. It means that such solution needs more than one broker instance in order to host a shard on each of them. Of course, a sender connected to one of these brokers can send messages to the shard hosted only on that broker. At same time, a receiver connected to a broker can get messages from the shard that is hosted on that broker and can not see available messages in the shards hosted on the other brokers, even if they are all parts of the same queue.

The big problem in this scenario, designed only with brokers, is that a receiver can be stucked on an empty shard without reading any messages while the shards on the other brokers have messages to deliver. it is a real problem because the receiver is interested in receiving messages from the whole queue and it does not take care if it is shared or not. Because of this problem, the receiver sees the queue as empty even if it is not so true due to the sharding and the messages available on the other shards.

The above problem can be solved adding a AMQ Interconnect instance in the network in front of the brokers and leverage on its waypoint feature with related auto-links.

Consider the following simple scenario :

Figure 11.5. Sharded Queue Enabled Topology

With such solution and connecting to the "Router.A", sender and receiver do not know anything about sharding; they want send and receive messages to/from the whole queue that is the only thing they are aware of. They are both connected to the router and see only one address (related to the queue).

The "Router.A" configuration is something like following.

router {
    mode: standalone
    id: Router.A
}

listener {
    host: 0.0.0.0
    port: 6000
    authenticatePeer: no
}

connector {
    name: BROKER1
    addr: 127.0.0.1
    port: 5672
    role: route-container
}

connector {
    name: BROKER2
    addr: 127.0.0.1
    port: 5673
    role: route-container
}

address {
    prefix: my_queue
    waypoint: yes
}

autoLink {
    addr: my_queue
    connection: BROKER1
    dir: in
}

autoLink {
    addr: my_queue
    connection: BROKER1
    dir: out
}

autoLink {
    addr: my_queue
    connection: BROKER2
    dir: in
}

autoLink {
    addr: my_queue
    connection: BROKER2
    dir: out
}

The router has a listener for incoming connection from clients and two connector instances in order to connect to both brokers. The whole queue is named my_queue hosted in terms of shards on both brokers and the router is configured with a waypoint for that address. Finally, there are two auto-links in both directions for that queue on both brokers.

Using the Python sample from the Qpid Proton library, the sender can connect to the "Router.A" and start to send messages to the queue; the console output is like following.

$ sudo python simple_send.py -a localhost:6000/my_queue -m 10
all messages confirmed

All messages are confirmed because they reached the queue and, thanks to the default balanced distribution on the address, the messages are delivered to both shards on the brokers (5 messages per shard). Using the qdstat tool on the router, the distribution is clear.

$ sudo qdstat -b localhost:6000 -l
Router Links
  type      dir  conn id  id  peer  class   addr                  phs  cap  undel  unsettled  deliveries  admin    oper
  =======================================================================================================================
  endpoint  in   1        6         mobile  my_queue              1    250  0      0          0           enabled  up
  endpoint  out  1        7         mobile  my_queue              0    250  0      0          5           enabled  up
  endpoint  in   2        8         mobile  my_queue              1    250  0      0          0           enabled  up
  endpoint  out  2        9         mobile  my_queue              0    250  0      0          5           enabled  up
  endpoint  in   8        19        mobile  $management           0    250  0      0          1           enabled  up
  endpoint  out  8        20        local   temp.qCGHruCa4UIvYrS       250  0      0          0           enabled  up

There are the out links (from router to brokers) for the my_queue address (id values 7 and 9) which have each 5 deliveries. It shows messages distributed across brokers and related shards for the queue; it is confirmed by the different connections they are tied (conn id values 1 and 2).

Starting the receiver connected to the "Router.A", it gets all the available messages from the queue.

$ sudo python simple_recv.py -a localhost:6000/my_queue -m 10
{u'sequence': 1L}
{u'sequence': 2L}
{u'sequence': 3L}
{u'sequence': 4L}
{u'sequence': 5L}
{u'sequence': 6L}
{u'sequence': 7L}
{u'sequence': 8L}
{u'sequence': 9L}
{u'sequence': 10L}

As for the sender, they are received through both the brokers and related shards. it is confirmed using the qdstat tool.

$ sudo qdstat -b localhost:6000 -l
Router Links
  type      dir  conn id  id  peer  class   addr                  phs  cap  undel  unsettled  deliveries  admin    oper
  =======================================================================================================================
  endpoint  in   1        6         mobile  my_queue              1    250  0      0          5           enabled  up
  endpoint  out  1        7         mobile  my_queue              0    250  0      0          5           enabled  up
  endpoint  in   2        8         mobile  my_queue              1    250  0      0          5           enabled  up
  endpoint  out  2        9         mobile  my_queue              0    250  0      0          5           enabled  up
  endpoint  in   10       22        mobile  $management           0    250  0      0          1           enabled  up
  endpoint  out  10       23        local   temp.HT+f3ZilGP5o3wo       250  0      0          0           enabled  up

There are the in links (from brokers to router) for the my_queue address (id values 6 and 8) which have each 5 deliveries. It shows messages distributed across brokers and related shards for the queue; it is confirmed by the different connections they are tied (conn id values 1 and 2).

One disadvantage of sharded queues is that the receiver might receive messages "out of order" even with very good performance.