ActiveMQ MQTT: unsustainable latency by subscribers on high-throughput message flows

Solution In Progress - Updated -

Issue

Here is our test setup:
Broker: The JBoss A-MQ 6.1 Broker is running on AWS in a EC2 m3.xlarge ubuntu 14.04 instance with OpenJDK.

Producers: 800 MQTT clients connect to the Broker, create ~40 subscriptions each, and start publishing messages of 1.5KB in size, at the frequency of 1 msg every 30 sec or so. Publishers are running in a local Ubuntu box connected to the Internet through Ethernet on our local network.

Consumers: 5 MQTT clients connect to the Broker, create ~10 subscriptions each matching all messages published by the Producers, and very rarely publish a larger message (10KB) randomly addressed to one of the Producers. Consumers are running in a local Ubuntu box connected to the Internet through Ethernet on our local network.

During the test run, the rate of messages received by the consumers is around 1/2 message rate of the publishers.
This causes a large accumulation of messages in the broker which will eventually become unsustainable.

The problem cannot be reproduced if the Consumers are moved to a machine in the AWS.

The problem cannot be reproduced if the mosquitto broker is used instead and there are NO changes in the test configuration;
same broker machine, same test drivers, same network. So this seems to exclude bandwidth problems.

Environment

  • JBoss A-MQ 6.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.