Exceptions during cache shutdown if the cluster view is changed at the same time in JDG6

Solution Verified - Updated -

Issue

During JDG shutdown, the following exception is thrown:

2015-12-21 14:22:56,966 INFO  [Incoming-1,NODE1-33639] jgroups.JGroupsTransport  - ISPN000094: Received new cluster view: [NODE2-49798|12] (3) [NODE2-49798, NODE1-33639, NODE3-49830]
2015-12-21 14:22:56,968 ERROR [Incoming-1,NODE1-33639] pbcast.GMS  - JGRP000027: failed passing message up
java.lang.RuntimeException: org.infinispan.commons.CacheListenerException: ISPN000280: Caught exception [java.util.concurrent.RejectedExecutionException] while invoking method [public void org.infinispan.transaction.TransactionTable.onViewChange(org.infinispan.notifications.cachemanagerlistener.event.ViewChangedEvent)] on listener instance: org.infinispan.transaction.TransactionTable@3d5ab0ba
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:682)
    at org.jgroups.JChannel.up(JChannel.java:733)
    at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1029)
    at org.jgroups.protocols.RSVP.up(RSVP.java:201)
    at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
    at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:732)
    at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:146)
    at org.jgroups.protocols.pbcast.GMS.up(GMS.java:922)
    at org.jgroups.stack.Protocol.up(Protocol.java:412)
    at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
    at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
    at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
    at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
    at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
    at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
    at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
    at org.jgroups.protocols.FD.up(FD.java:255)
    at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:297)
    at org.jgroups.protocols.MERGE3.up(MERGE3.java:288)
    at org.jgroups.protocols.Discovery.up(Discovery.java:291)
    at org.jgroups.protocols.TP.passMessageUp(TP.java:1572)
    at org.jgroups.protocols.TP$MyHandler.run(TP.java:1791)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.infinispan.commons.CacheListenerException: ISPN000280: Caught exception [java.util.concurrent.RejectedExecutionException] while invoking method [public void org.infinispan.transaction.TransactionTable.onViewChange(org.infinispan.notifications.cachemanagerlistener.event.ViewChangedEvent)] on listener instance: org.infinispan.transaction.TransactionTable@3d5ab0ba
    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:287)
    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)
    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocationImpl.invoke(AbstractListenerImpl.java:305)
    at org.infinispan.notifications.cachemanagerlistener.CacheManagerNotifierImpl.notifyViewChange(CacheManagerNotifierImpl.java:88)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport$NotifyViewChange.emitNotification(JGroupsTransport.java:638)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.viewAccepted(JGroupsTransport.java:708)
    at org.jgroups.blocks.MessageDispatcher.handleUpEvent(MessageDispatcher.java:602)
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:679)
    ... 25 more
Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1f5986a3 rejected from java.util.concurrent.ScheduledThreadPoolExecutor@2e964769[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1696]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
    at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
    at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:546)
    at java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:646)
    at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:641)
    at org.infinispan.transaction.TransactionTable.onViewChange(TransactionTable.java:491)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:282)
    ... 32 more

Environment

  • Red Hat JBoss Data Grid (JDG)
    • 6.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In