Consumers are being dropped and re-added to the queue constantly
Environment
- Fuse Message Broker all versions
Issue
JMS Consumers are being dropped and re-added to the queue constantly
With the tracing configured to DEBUG you see a pattern similar to below, where a consumer is added and then soon after is removed:
2009-02-18 14:43:10,699 [127.0.0.1:37328] DEBUG WireFormatNegotiator - Received WireFormat: WireFormatInfo { version=2, properties={CacheSize=1024, CacheEnabled=true, SizePrefixDisabled=false,
TcpNoDelayEnabled=true, MaxInactivityDuration=30000, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
2009-02-18 14:43:10,699 [localhost:61616] DEBUG WireFormatNegotiator - Sending: WireFormatInfo { version=2, properties={CacheSize=1024, CacheEnabled=true, SizePrefixDisabled=false,
TcpNoDelayEnabled=true, MaxInactivityDuration=30000, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
2009-02-18 14:43:10,700 [127.0.0.1:37328] DEBUG WireFormatNegotiator - tcp:///127.0.0.1:37328 before negotiation: OpenWireFormat{version=2, cacheEnabled=false, stackTraceEnabled=false,
tightEncodingEnabled=false, sizePrefixDisabled=false}
2009-02-18 14:43:10,703 [127.0.0.1:37326] DEBUG TransportConnection - Stopped connection: /127.0.0.1:37326
2009-02-18 14:43:10,703 [127.0.0.1:37328] DEBUG WireFormatNegotiator - tcp:///127.0.0.1:37328 after negotiation: OpenWireFormat{version=2, cacheEnabled=true, stackTraceEnabled=true,
tightEncodingEnabled=true, sizePrefixDisabled=false}
2009-02-18 14:43:10,704 [127.0.0.1:37328] DEBUG TransportConnection - Setting up new connection: org.apache.activemq.broker.jmx.ManagedTransportConnection@4fdf11
2009-02-18 14:43:10,741 [127.0.0.1:37328] DEBUG AbstractRegion - Adding consumer: ID:plucky600-49099-1234986189476-0:5:-1:1
2009-02-18 14:43:10,797 [127.0.0.1:37328] DEBUG AbstractRegion - Removing consumer: ID:plucky600-49099-1234986189476-0:5:-1:1
2009-02-18 14:43:10,800 [127.0.0.1:37328] DEBUG TransportConnection - Stopping connection: /127.0.0.1:37328
2009-02-18 14:43:10,801 [127.0.0.1:37328] DEBUG TransportConnection - Stopped connection: /127.0.0.1:37328
2009-02-18 14:43:10,806 [127.0.0.1:37330] DEBUG WireFormatNegotiator - Received WireFormat: WireFormatInfo { version=2, properties={CacheSize=1024, CacheEnabled=true, SizePrefixDisabled=false,
TcpNoDelayEnabled=true, MaxInactivityDuration=30000, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
2009-02-18 14:43:10,806 [localhost:61616] DEBUG WireFormatNegotiator - Sending: WireFormatInfo { version=2, properties={CacheSize=1024, CacheEnabled=true, SizePrefixDisabled=false,
TcpNoDelayEnabled=true, MaxInactivityDuration=30000, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
2009-02-18 14:43:10,808 [127.0.0.1:37330] DEBUG WireFormatNegotiator - tcp:///127.0.0.1:37330 before negotiation: OpenWireFormat{version=2, cacheEnabled=false, stackTraceEnabled=false,
tightEncodingEnabled=false, sizePrefixDisabled=false}
2009-02-18 14:43:10,808 [127.0.0.1:37330] DEBUG WireFormatNegotiator - tcp:///127.0.0.1:37330 after negotiation: OpenWireFormat{version=2, cacheEnabled=true, stackTraceEnabled=true
tightEncodingEnabled=true, sizePrefixDisabled=false}
2009-02-18 14:43:10,809 [127.0.0.1:37330] DEBUG TransportConnection - Setting up new connection: org.apache.activemq.broker.jmx.ManagedTransportConnection@179c7cf
2009-02-18 14:43:10,849 [127.0.0.1:37330] DEBUG AbstractRegion - Adding consumer: ID:plucky600-49099-1234986189476-0:6:-1:1
2009-02-18 14:43:10,900 [127.0.0.1:37330] DEBUG AbstractRegion - Removing consumer: ID:plucky600-49099-1234986189476-0:6:-1:1
The important lines from the above trace are these two:
2009-02-18 14:43:10,849 [127.0.0.1:37330] DEBUG AbstractRegion - Adding consumer: ID:plucky600-49099-1234986189476-0:6:-1:1
2009-02-18 14:43:10,900 [127.0.0.1:37330] DEBUG AbstractRegion - Removing consumer: ID:plucky600-49099-1234986189476-0:6:-1:1
Resolution
Ensure that JMS resources such as JMS connections, JMS sessions and JMS producer/consumers are reused as much as possible in the lifetime of the JMS client application.
For example the following ProducerTool (from examples/src/ProducerTool.java) should be kept outside of the for loop:
public static void main(String[] args)
{
ProducerTool producerTool = new ProducerTool();
for ( int c = 0; c < 10; c++)
{
String[] unknonwn = CommandLineSupport.setOptions(producerTool, args);
if( unknonwn.length > 0 ) {
System.out.println("Unknown options: " + Arrays.toString(unknonwn));
System.exit(-1);
}
producerTool.run();
}
}
In which case the JMS connection objects that are instantiated from within the constructor of ProducerTool will stay in scope with each iteration of the for loop. There is no need to recreate any of the JMS resources.
Root Cause
The JMS client is letting the objects related to the connection to the broker fall out of scope. As an example let's say the producer in examples/src/ProducerTool.java is modified like the following:
public static void main(String[] args) {
for (int c = 0; c < 10; c++)
{
ProducerTool producerTool = new ProducerTool();
String[] unknonwn = CommnadLineSupport.setOptions(producerTool, args);
if( unknonwn.length > 0 ) {
System.out.println("Unknown options: " + Arrays.toString(unknonwn));
System.exit(-1);
}
producerTool.run();
}
}
In this case the JMS connection is created within ProducerTool. ProducerTool will be garbage collected at the end of each iteration of the for loop. With each garbage collection all references to the JMS connection will also be garbage collected. This will cause the producer to drop it's connection to the broker.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments