Frequently Asked Questions

JBoss Operations Network 3.1.1

common questions, issues, and fixes

JBoss Operations Network Team

June 12, 2012, updated October 3, 2012

Abstract

These questions cover some common issues, setup information, requirements, and errors for JBoss ON.

1. General

Q: What is the difference between JBoss Operations Network and RHQ?
Q: Is there a publicly available issue tracker system to search for bugs and submit enhancement requests?
Q: What databases are supported?
Q: Why can't I start JBoss ON with Java 5?
Q: How can I find what my user preferences are?
Q: What is the syntax for regular expressions used within JBoss ON?
Q: How often does JBoss ON check the availability of resources?
Q: Why is the JBoss ON agent waiting at startup?
Q: How do I install a supported version of PostgreSQL on Red Hat Enterprise Linux?
Q: How can I run SQL commands against the JBoss ON database from the JBoss ON console?
Q: Is JBoss ON supported on VMWare?
Q: To help debug Out Of Memory conditions, how do I get the agent or server to dump heap when it runs out of memory or on demand?
Q:
What is the difference between JBoss Operations Network and RHQ?
A:
RHQ is the upstream, open source version of JBoss Operations Network. JBoss Operations Network is a commercial product which is built on RHQ, with extensive testing and official customer support through Red Hat.
Q:
Is there a publicly available issue tracker system to search for bugs and submit enhancement requests?
A:
To search for a bug, report a bug, or submit an enhancement request, use Bugzilla. Select the Other distribution and then the RHQ Project component.
Q:
What databases are supported?
A:
PostgreSQL 8.2.4 and higher 8.2.x versions; all releases of PostgreSQL 8.3, 8.4, and 9.0; and Oracle 11 are supported databases.
The full list of supported databases, server and agent platforms, and Java versions is available at http://www.redhat.com/jboss_on/requirements.
Q:
Why can't I start JBoss ON with Java 5?
A:
Java 5 is no longer supported. Upgrade to Java 6.
The full list of supported databases, server and agent platforms, and Java versions is available at http://www.redhat.com/jboss_on/requirements.
Q:
How can I find what my user preferences are?
A:
In either the database client or from the http://server.hostname:7080/admin/test/sql.jsp page, run this SQL command:
select id, name, string_value 
from rhq_config_property 
where configuration_id = (select configuration_id 
from rhq_subject 
where name = 'your-user-name')
Q:
What is the syntax for regular expressions used within JBoss ON?
A:
JBoss ON uses regular expressions in several places in both the UI and in configuration files. All of the regular expressions follow the standard Java format, as covered in the Sun documentation for regex syntax and date/time format syntax.
Q:
How often does JBoss ON check the availability of resources?
A:
Agents check for resource available every minute for servers and every 10 minutes for services. The agent itself runs an availability check every 30 seconds, on a subset of its total inventory; this keeps a steady load on the agent and prevents memory-intensive usage spikes. The agent sends the results to the JBoss ON server.
The frequency that a specific resource is checked for availability is scheduled, much like scheduling the frequency for a metric. This availability check interval can be changed for a resource on its Monitoring > Schedules tab.
The frequency that the agent itself runs an availability check is defined in the rhq.agent.plugins.availability-scan.period-secs setting. The default is 30 seconds. For performance reasons, it should never be lower than 30 seconds. It is possible to extend the scan interval by setting a new interval as one of the ADDITIONAL_JAVA_OPTIONS values. For example:
RHQ_AGENT_ADDITIONAL_JAVA_OPTS="-Drhq.agent.plugins.availability-scan.period-secs=45"
Q:
Why is the JBoss ON agent waiting at startup?
A:
Sometimes when you start the JBoss ON agent from the command line, it will not proceed to the sending> prompt, but sit there and wait. There are several possible reasons for this.
The server has rejected the agent registration request.

If the agent returns this message at start up, it means that the agent is known to the server under one name but is sending a different name when it starts:

Cause: [org.rhq.core.clientapi.server.core.AgentRegistrationException:The 
agent asking for registration is trying to register the same address/port 
[172.31.7.7:16163] that is already registered under a different name [example]; 
if this new agent is actually the same as the original, then re-register with
the same name]
To solve this, start the agent with option --clean and give the correct name.
The agent cannot reach the server.

This is an agent state where the server cannot be reached because the server is down or because a firewall has blocked the traffic. Make sure port 7080 on the server machine is reachable from the agent's machine. You can check this through a web browser.

The server cannot connect to the agent.

An error saying that the server cannot ping the agent's endpoint means that the agent can communicate with the server, but the server cannot communicate with the agent. This may mean that the agent port is blocked by a firewall.

The server has rejected the agent registration request. Cause: 
[org.rhq.core.clientapi.server.core.AgentRegistrationException:Server cannot 
ping the agent's endpoint. The agent's endpoint is probably invalid or there 
is a firewall preventing the server from connecting to the agent. Endpoint:
socket://172.31.7.3:12345/....
The agent does not have plug-ins - it will now wait for them to be downloaded.

This usually means that the server has a different security token than the one the agent was sending. This could have resulted from the java preferences entry being mangled, for example, by testing with different agent versions or VMs.

You will see this message only on initial agent startup when it does not have any plug-ins. If plug-ins were downloaded in a previous run, you will probably run in the situation shown below.
If you see this on the agent, you should also see messages like this on the server:
11:40:48,454 WARN [CommandProcessor] {CommandProcessor.failed-
authentication}Command failed to be authenticated! This command will be 
ignored and not processed: Command: type=[remotepojo]; cmd-in-response=
[false]; config=[{rhq.security-token=1217855913569-109582636-403140853869881172, rhq.send-throttle=true}];
params=
[{targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService, 
invocation=NameBasedInvocation[getLatestPlugins]}]
To solve this, start the agent interactively with the --clean option.
Agent startup is OK, but ping command fails.

Here, the agent successfully starts, but there may be other agent communication problems, like no monitoring data are sent. Trying to ping the agent command line can return an error like the following:

sending> ping
	
	
Pinging...
	
	
Failed to execute prompt command [ping]. Cause: 
org.rhq.enterprise.communications.command.server.AuthenticationException:Command 
failed to be authenticated! This command will be ignored and not processed: 
Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-
token=1214208960346-102975580-7334156733284942657, rhq.send-throttle=true}]; 
params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping, 
invocation=NameBasedInvocation[ping]}]
The server log will probably contain several CommandProcessor.failed-authentication messages. This means that the agent's port is probably blocked by a firewall so that the server cannot communicate with it.
The forward and backward mappings of the IP address for high availability don't match.

Make sure the IP address of your computer can be reverse-mapped to the computer name, and that this name maps back to the same IP address. This needs to be true for all your hosts.

For example, if you have an IP address of 172.31.7.7, then the results of name resolution will look like the following:
$ dig -x 172.31.7.7
[...]
;; ANSWER SECTION:
7.7.31.172.in-addr.arpa. 86400 IN     PTR     example
$
$ dig example
[...]
;; ANSWER SECTION:
example       74030   IN      A       172.31.7.7
If your agent-server communication was working and it stopped suddenly, go to the JBoss ON server UI. Then go to Administration > High Availability > Server and check if the name displayed matches what you expect. Check the same for the agents.
If this all works, and the agent is still hanging, there may be other possibilities for misconfiguration.
Q:
How do I install a supported version of PostgreSQL on Red Hat Enterprise Linux?
A:
The default PostgreSQL version on Red Hat Enterprise Linux 5.5 is an older, unsupported version of PostgreSQL, so it has to be updated.
To install Postgres from Red Hat Network:
  1. Log into http://rhn.redhat.com with your RHN/JBoss credentials.
  2. Add the Red Hat Application Stack v2 channel.
  3. Update the system:
    sudo yum update
  4. Then update PostgreSQL specifically:
    sudo yum install postgresql-server
    The data directory is installed in /var/lib/pgsql/data. JBoss ON supports PostgreSQL 8.2.4 and later 8.2.x versions and all releases of PostgreSQL 8.3, 8.4, and 9.0.
  5. Install and configure the JBoss ON server as normal.
Q:
How can I run SQL commands against the JBoss ON database from the JBoss ON console?
A:
Go to the SQL page:
http://server.example.com:7080/admin/test/sql.jsp
Q:
Is JBoss ON supported on VMWare?
A:
VMWare ESX is supported by Red Hat Enterprise Linux as equivalent to bare metal. In the case of a hardware issue with a VMWare product, work with VMWare support to resolve the issue.
JBoss EAP is fully supported on all current versions of Red Hat Enterprise Linux. Support levels and SLA will vary depending on the entitlements you purchased.
Q:
To help debug Out Of Memory conditions, how do I get the agent or server to dump heap when it runs out of memory or on demand?
A:
Pass these JVM arguments to the server or agent (setting the RHQ_AGENT_ADDITIONAL_JAVA_OPTS or RHQ_SERVER_ADDITIONAL_JAVA_OPTS variables).
-XX:+HeapDumpOnOutOfMemoryError -XX:+HeapDumpOnCtrlBreak
To drop the heap dump file in a particular location, add a path:
-XX:HeapDumpPath=location
See the SUN JVM Debugging Options for more info.