High CPU and heap overhead in JMX when many rars are deployed
Issue
- We have many (hundreds) of rars deployed. We noticed substantial increased heap usage in this case and CPU is persistently high when we connect via JMX and mbeans are never rendered when viewed via jconsole. The heap dump shows the org.jboss.as.connector.subsystems.resourceadapters.IronJacamarResources are quite large. And thread dumps show CPU is high in calls like the following:
"RMI TCP Connection(1)-127.0.0.1" #100 daemon prio=9 os_prio=0 tid=0x00007f02240c5000 nid=0x3a43 runnable [0x00007f024d6f7000]
java.lang.Thread.State: RUNNABLE
at java.security.AccessController.getStackAccessControlContext(Native Method)
at java.security.AccessController.getContext(AccessController.java:820)
at org.jboss.as.controller.SecurityActions.getCaller(SecurityActions.java:132)
at org.jboss.as.controller.AbstractOperationContext.getCaller(AbstractOperationContext.java:1017)
at org.jboss.as.controller.OperationContextImpl.authorize(OperationContextImpl.java:1284)
at org.jboss.as.controller.OperationContextImpl.readResourceFromRoot(OperationContextImpl.java:646)
at org.jboss.as.controller.OperationContextImpl.readResourceFromRoot(OperationContextImpl.java:632)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.addParentResource(ReadResourceDescriptionHandler.java:786)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getAllActualResourceAddresses(ReadResourceDescriptionHandler.java:774)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getAllActualResourceAddresses(ReadResourceDescriptionHandler.java:778)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getAllActualResourceAddresses(ReadResourceDescriptionHandler.java:778)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getAllActualResourceAddresses(ReadResourceDescriptionHandler.java:778)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getAllActualResourceAddresses(ReadResourceDescriptionHandler.java:778)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getAllActualResourceAddresses(ReadResourceDescriptionHandler.java:778)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.getLocalResourceAddresses(ReadResourceDescriptionHandler.java:713)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.initLocalResourceAddresses(ReadResourceDescriptionHandler.java:702)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler$ReadResourceDescriptionAccessControlContext.access$200(ReadResourceDescriptionHandler.java:689)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler.doExecuteInternal(ReadResourceDescriptionHandler.java:204)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler.doExecute(ReadResourceDescriptionHandler.java:162)
at org.jboss.as.controller.operations.global.ReadResourceDescriptionHandler.execute(ReadResourceDescriptionHandler.java:155)
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:702)
at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:537)
at org.jboss.as.controller.AbstractOperationContext.completeStepInternal(AbstractOperationContext.java:338)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:314)
at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1144)
at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:331)
at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:188)
at org.jboss.as.jmx.model.ResourceAccessControlUtil.getResourceAccess(ResourceAccessControlUtil.java:85)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:51)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
at org.jboss.as.jmx.model.RootResourceIterator.iterate(RootResourceIterator.java:43)
Environment
- JBoss Enterprise Application Platform (EAP) 6.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.