7.6. Enabling Debug Mode for the Agent

The JBoss ON agent, like the JBoss ON server, uses log4j for its logging. To troubleshoot agent performance or server-agent communication, enable debug logging for the agent, which enables the log4j debug log.
The log files are in the agentRoot/rhq-agent/logs directory.

7.6.1. Using an Environment Variable

The quickest way to enable debug logging is to set the RHQ_AGENT_DEBUG environment variable to any value before starting the agent. When you start the agent, both the launcher scripts and the agent itself will output debug messages.
If the JBoss ON agent is running on Microsoft Windows using the service wrapper, set RHQ_AGENT_DEBUG and then install the service:
rhq-agent-wrapper.bat install

7.6.2. Setting log4j Priorities

log4j categories support priorities for logging levels. This means that different areas of the agent can be configured for different log levels.
Do not set the RHQ_AGENT_DEBUG environment variable if you are setting priorities in the log4j.xml file. The environment variable overrides this log4j.xml configuration.
To enable debug logging for a category, change the priority value to DEBUG:
  1. Open the agent log4j file:
    # vim agentRoot/rhq-agent/conf/log4j.xml
  2. Reset the priority element for the category. By default, the agent configuration has logging for both incoming and outgoing server-agent communication and for the base org.rhq class. Optionally, logging can be enabled for plug-in class loaders and JBoss remoting communication.
       <!-- ================ -->
       <!-- Limit categories -->
       <!-- ================ -->
       <!-- RHQ -->
       <category name="org.rhq">
          <priority value="INFO"/>
       <!-- RHQ outgoing command tracing  - set to TRACE to trace commands sent by the agent -->
       <category name="org.rhq.enterprise.communications.command.client.OutgoingCommandTrace">
          <priority value="NONE"/>
          <appender-ref ref="COMMANDTRACE"/>
  3. Restart the agent to load the new configuration.
The log4j file format is described more in the Apache log4j documentation.

7.6.3. Using the Agent debug Prompt Command

Debug logging can be enabled using the debug command in the agent command prompt (Section 7.3.1, “Opening the Agent Command Prompt”).
Using the --enable option enables the log4j debug log.
> debug --enable
log4j:WARN No appenders could be found for logger (org.rhq.core.pc.measurement.MeasurementCollectorRunner).
log4j:WARN Please initialize the log4j system properly.
Switched to log file [log4j-debug.xml]. Root log level is [DEBUG]
To enable debug logging specifically for server-agent communication layers, set the --comm option to true.
> debug --comm=true
Agent-server communications tracing has been enabled.
You may set the following, additional configuration settings
to collect more detailed trace data. You can set these
using the setconfig prompt command. Please refer to the
documentation for more information on these settings. The
values you see here are the current settings:
The debug command can also be used to check all of the agent threads, to the server and to the system management handlers, using the --threaddump option. This prints the information for each thread, whether the thread is running or any errors that the agent is encountering, per thread. For example:
> debug --threaddump
"DestroyJavaVM" Id=47 RUNNABLE

"RHQ Agent Prompt Input Thread" Id=46 RUNNABLE

"EventManager.sender-2" Id=49 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@17d7c01
        at sun.misc.Unsafe.park(Native Method)
        -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@17d7c01
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)