ActiveMQ MQTT: unsustainable latency by subscribers on high-throughput message flows
Issue
Here is our test setup:
Broker: The JBoss A-MQ 6.1 Broker is running on AWS in a EC2 m3.xlarge ubuntu 14.04 instance with OpenJDK.
Producers: 800 MQTT clients connect to the Broker, create ~40 subscriptions each, and start publishing messages of 1.5KB in size, at the frequency of 1 msg every 30 sec or so. Publishers are running in a local Ubuntu box connected to the Internet through Ethernet on our local network.
Consumers: 5 MQTT clients connect to the Broker, create ~10 subscriptions each matching all messages published by the Producers, and very rarely publish a larger message (10KB) randomly addressed to one of the Producers. Consumers are running in a local Ubuntu box connected to the Internet through Ethernet on our local network.
During the test run, the rate of messages received by the consumers is around 1/2 message rate of the publishers.
This causes a large accumulation of messages in the broker which will eventually become unsustainable.
The problem cannot be reproduced if the Consumers are moved to a machine in the AWS.
The problem cannot be reproduced if the mosquitto broker is used instead and there are NO changes in the test configuration;
same broker machine, same test drivers, same network. So this seems to exclude bandwidth problems.
Environment
- JBoss A-MQ 6.1
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.