JON fails to monitor JBoss instance, with java.lang.NoClassDefFoundError

Solution Verified - Updated -

Environment

  • JBoss Enterprise Application Platform (EAP) 4.3
  • JBoss Operation Network (JON)

Issue

The JON Console is unable to connect to the remote JBOSS EAP JVM. After going to Servers > JBOSS EAP > Inventory > connection and selecting edit, updating the jmx user credentails and saving the follwoing error occurs:

org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=504106, type=JBossAS Server, key=/var/lib/jbossas/server/default, name=host.example.com JBossEAP 4.3.0.GA_CP01 default (1099), parent=host.example.com, version=4.3.0.GA_CP01].
    at org.rhq.core.pc.inventory.InventoryManager.activateResource(InventoryManager.java:1141)
    at org.rhq.core.pc.inventory.InventoryManager.refreshResourceComponentState(InventoryManager.java:2072)
    at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:1879)
    at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:1885)
    at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:734)
    at org.rhq.core.pc.inventory.InventoryManager.handleReport(InventoryManager.java:714)
    at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.call(AutoDiscoveryExecutor.java:118)
    at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.run(AutoDiscoveryExecutor.java:92)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
    at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.Exception: java.lang.NoClassDefFoundError
    at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:488)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
    at java.util.concurrent.FutureTask.run(FutureTask.java:123)
    ... 3 more
Caused by: java.lang.NoClassDefFoundError
    at org.jboss.security.jndi.SecurityAssociationActions$SetPrincipalInfoAction.run(SecurityAssociationActions.java:72)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.jboss.security.jndi.SecurityAssociationActions.setPrincipalInfo(SecurityAssociationActions.java:110)
    at org.jboss.security.jndi.JndiLoginInitialContextFactory.getInitialContext(JndiLoginInitialContextFactory.java:105)
    at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
    at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:247)
    at javax.naming.InitialContext.init(InitialContext.java:223)
    at javax.naming.InitialContext.<init>(InitialContext.java:197)
    at org.mc4j.ems.impl.jmx.connection.support.providers.JBossConnectionProvider.getInitialContext(JBossConnectionProvider.java:131)
    at org.mc4j.ems.impl.jmx.connection.support.providers.JBossConnectionProvider.doConnect(JBossConnectionProvider.java:68)
    at org.mc4j.ems.impl.jmx.connection.support.providers.AbstractConnectionProvider.connect(AbstractConnectionProvider.java:99)
    at org.rhq.plugins.jbossas.JBossASServerComponent.loadConnection(JBossASServerComponent.java:1167)
    at org.rhq.plugins.jbossas.JBossASServerComponent.start(JBossASServerComponent.java:252)
    at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:484)
    ... 5 more

Resolution

  • Fix client/log4j.jar to be the version that ships with JBoss not be older one - for example copy server/default/lib/log4j.jar

Root Cause

client/log4j.jar had been replaced with an older version

Diagnostic Steps

  • Add RHQ_AGENT_DEBUG=true to  bin/rhq-agent-env.sh

  • See if there is any exceptions being thrown from SecurityAssociation.<clinit>

    Caused by: java.lang.NoSuchFieldError: TRACE
     at org.jboss.logging.Log4jLoggerPlugin.isTraceEnabled(Log4jLoggerPlugin.java:85)
     at org.jboss.logging.Logger.isTraceEnabled(Logger.java:122)
     at org.jboss.security.SecurityAssociation.<clinit>(SecurityAssociation.java:145)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments