Exceptions during cache shutdown if the cluster view is changed at the same time in JDG6

Solution Verified - Updated -

Issue

During JDG shutdown, the following exception is thrown:

2015-12-21 14:22:56,966 INFO  [Incoming-1,NODE1-33639] jgroups.JGroupsTransport  - ISPN000094: Received new cluster view: [NODE2-49798|12] (3) [NODE2-49798, NODE1-33639, NODE3-49830]
2015-12-21 14:22:56,968 ERROR [Incoming-1,NODE1-33639] pbcast.GMS  - JGRP000027: failed passing message up
java.lang.RuntimeException: org.infinispan.commons.CacheListenerException: ISPN000280: Caught exception [java.util.concurrent.RejectedExecutionException] while invoking method [public void org.infinispan.transaction.TransactionTable.onViewChange(org.infinispan.notifications.cachemanagerlistener.event.ViewChangedEvent)] on listener instance: org.infinispan.transaction.TransactionTable@3d5ab0ba
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:682)
    at org.jgroups.JChannel.up(JChannel.java:733)
    at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1029)
    at org.jgroups.protocols.RSVP.up(RSVP.java:201)
    at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
    at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:732)
    at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:146)
    at org.jgroups.protocols.pbcast.GMS.up(GMS.java:922)
    at org.jgroups.stack.Protocol.up(Protocol.java:412)
    at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
    at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
    at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
    at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
    at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
    at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
    at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
    at org.jgroups.protocols.FD.up(FD.java:255)
    at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:297)
    at org.jgroups.protocols.MERGE3.up(MERGE3.java:288)
    at org.jgroups.protocols.Discovery.up(Discovery.java:291)
    at org.jgroups.protocols.TP.passMessageUp(TP.java:1572)
    at org.jgroups.protocols.TP$MyHandler.run(TP.java:1791)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.infinispan.commons.CacheListenerException: ISPN000280: Caught exception [java.util.concurrent.RejectedExecutionException] while invoking method [public void org.infinispan.transaction.TransactionTable.onViewChange(org.infinispan.notifications.cachemanagerlistener.event.ViewChangedEvent)] on listener instance: org.infinispan.transaction.TransactionTable@3d5ab0ba
    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:287)
    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)
    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocationImpl.invoke(AbstractListenerImpl.java:305)
    at org.infinispan.notifications.cachemanagerlistener.CacheManagerNotifierImpl.notifyViewChange(CacheManagerNotifierImpl.java:88)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport$NotifyViewChange.emitNotification(JGroupsTransport.java:638)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.viewAccepted(JGroupsTransport.java:708)
    at org.jgroups.blocks.MessageDispatcher.handleUpEvent(MessageDispatcher.java:602)
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:679)
    ... 25 more
Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1f5986a3 rejected from java.util.concurrent.ScheduledThreadPoolExecutor@2e964769[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1696]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
    at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
    at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:546)
    at java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:646)
    at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:641)
    at org.infinispan.transaction.TransactionTable.onViewChange(TransactionTable.java:491)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:282)
    ... 32 more

Environment

  • Red Hat JBoss Data Grid (JDG)
    • 6.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content