Red Hat Training
A Red Hat training course is available for Red Hat JBoss Operations Network
for release information
Copyright © 2012 Red Hat, Inc.
1. New Features in JBoss Operations Network 3.1
1.1. New: JBoss Resource Plug-ins for JBoss Enterprise Application Platform 6
- A new architecture for managed domains, which introduce the idea of centralized domain controllers, local system host controllers, and low-level managed servers, with configuration defined in concentric levels from the domain down to local applications.A classic standalone server still exists that parallels the JBoss EAP 4 and 5 servers, which can be clustered or configured for high availability.
- Classes are modular, so classes can be loaded and unloaded as needed, lowering the EAP footprint and improving system performance.
- Define and manage application workflows. Test an application on a single server and then promote the application to a server group or deploy it on a new domain, a standalone server, or a cluster through a single provisioning path.
- Manage multiple standalone instances and EAP 6 domains from a single management console.
- Gain complete, chronological visibility into availability, performance metrics, auditable operations, events, and configuration changes.
- Organize EAP 6 resources in a platform-oriented hierarchy that offers a deployment-focused perspective on servers, services, and child applications.For example, a datasource in the EAP 6 console has three separate resources — the datasource definitions and then two separate views for metrics and configuration. In JBoss ON, this is consolidated into a single view for the datasource.
- Automatically detect runtime environment variables and launch script arguments. These start options can be viewed and edited through JBoss ON, which makes them easy to apply to other managed resources.
1.2. Enahanced: More Control over the Start Script Environment
- The start script used, including custom start scripts.
- Required environment variables set in the
run.conffile or the parent process (a subset of possible or all environment variables).
- Any arguments passed with the start script itself.
- EAP 6 only. What user the script is running as. If the EAP server and JBoss ON agent users are different, JBoss ON assigns a
sudocommand as a prefix command to use with the start script, such as
sudo -u jboss -g jboss.
1.3. Enhanced: More Responsive, Accurate, and Granular Availability Monitoring
One of the biggest changes to availability is allowing availability to be scheduled at different intervals for different resources and resource types, much like a metric schedule. This allows important resources to be prioritized higher than others.
- Server availability is collected every minute.
- Service availability is collected every 10 minutes.
- Some dependent child resources have availability checking disabled entirely, and they default to the parent availability state. These resource can have availability checking enabled if desired.
A resource can be in several different states: UP, DOWN, UNKNOWN, and DISABLED.
Table 1. Group Availability States
|If the Resource States Are ....||... the Group State Is ...|
|Empty Group (Unknown)||Empty|
|All Red (Down)||Red (Down)|
|Some Down or Unknown||Yellow (Mixed)|
|Some Orange (Disabled)||Orange (Disabled)|
|All Green (Up)||Green (Up)|
Previously, JBoss ON could send an alert on an availability state change, whether a resource went up or down. Alerting based on availability has been enhanced in two ways:
- A new not up option has been added for availability state changes. An alert can now be fired if a resource comes up, goes down, or goes into another (not up) state.
- A new alert condition has been added for availability duration. If a resource changes state and then stays in that state for a given amount of time, an alert is fired.
avail command for the agent command prompt can send an availability report that contains only changed availability states or all current states, regardless of changes. This has always been true.
availcommand checked every single resource in the agent's inventory. Now, it only checks the next scheduled resource by default. Checking the complete inventory requires the
The agent heartbeat (the signal to the server that the agent is running) has been removed from general availability reporting. Instead, the agent sends a lightweight ping to the server.
1.4. New: Export Reports to CSV
1.5. Tech Preview: REST Interface
1.6. Enhanced: Additional Monitoring Metrics
- A new trait has been added to datasource and connection factory monitoring metrics for JBoss AS 5 servers which determines whether the connection is available. This tests the connection settings every 15 minutes, by default.
- A new count for the number of open files for a JVM process has been added. Operating systems commonly put limits on how many open files a single process can have, which can cause otherwise performant JVM processes to fail when they hit that limit. Tracking the open files metric can help signal administrators when the JVM is approaching that limit.This metric is only relevant on Linux and Unix systems; it is not available on Windows systems.
- New actual free memory and actual total memory metrics have been added for platforms. These memory counts include cache and buffer settings in the available memory calculation.
- A new maximum thread count metric has been added to the embedded JBossWeb Connector resource for JBoss AS 5 servers.
1.7. New: Access Control Permission to View User Details
1.8. Enhanced: Better Discovery and Management for Custom JVM Resources
- JMX remoting is enabled, and the remoting port is set through the command line, so that the agent can read the options. This is the method that has always been used to discover JMX servers.
- The JVM process is accessible through the Java attach API and has a resource key system property set in its command line. The key allows the agent to identify the appropriate Java process and provide a unique resource identity.In this case, the agent and the JVM must be running as the same system user, or the agent cannot attach to the process.
1.9. New: Resource Plug-in to Monitor and Manage JBoss Data Grid (Infinispan)
1.10. Enhanced: Additional Information in SNMP Notifications
1.11. New: Send the Current JBoss ON Server State to the Logs
2012-05-14 19:44:28,587 INFO [SystemInfoManager] SystemInformation: ******** CAM_LDAP_BIND_PW: [- non null -] AlertDefinitionCount:  CAM_LDAP_BASE_DN: [o=JBoss,c=US] AVAILABILITY_PURGE:  CAM_JAAS_PROVIDER: [false] BuildNumber: [ca099bc:3a46aff] ServerCount:  DATABASE_DRIVER_NAME: [PostgreSQL Native Driver] RESOURCE_GENERIC_PROPERTIES_UPGRADE: [false] ... 8< ...
2. Structural Changes in JBoss ON 3.1
2.1. General Changes to the Remote API
2.2. Changes in the Resource Live Availability Method
ResourceManagerRemote.getLiveResourceAvailability()method now properly returns
2.3. Obfuscated Passwords for Content Providers
2.4. Deprecated Plug-ins
- The JBoss AS/EAP 4 resource plug-in is deprecated in JBoss ON 3.1. Support will be removed in a later release, coincidental with the sunset of JBoss AS 4.
- The PostgreSQL database resource plug-in is deprecated in JBoss ON 3.1. Support will be removed in a later release.
- The Generic Database resource plug-in is deprecated in JBoss ON 3.1. Support will be removed in a later release.
defaultattribute for the
c:optionelement in agent plugin-descriptors is deprecated and will be removed in a future JBoss ON version.
2.5. Delayed Plug-In Releases for JBoss Enterprise Web Server and Tomcat
- JBoss Enterprise Web Server
2.6. Deprecated Webservices Class in the Remote API
3. JBoss Operations Network Prerequisites and Basic Installation
3.1. Supported Platforms
- Red Hat Enterprise Linux 6 (32-bit and 64-bit)
- Red Hat Enterprise Linux 5 (32-bit and 64-bit)
- Red Hat Enterprise Linux 4 (32-bit and 64-bit)
- Microsoft Windows 2008 R2 (32-bit and 64-bit)
- Solaris 11 (64-bit)
- Solaris 10 (64-bit) (deprecated)
3.2. Supported Databases
- Oracle 11g
- PostgreSQL 9.0
- PostgreSQL 8.4.x
- PostgreSQL 8.3.x
- PostregSQL 8.2.4 and later releases (deprecated)
3.3. Hardware Requirements
3.4. Required Java Versions
3.5. Supported Web Browsers
- Firefox 10
- Internet Explorer 8
3.6. Quick Setup for JBoss Operations Network 3.1
- Set up a database named
rhqadminand a database user named
rhqadmin. Make sure the database user has adequate permissions to perform operations on the JBoss ON database.For more detailed database setup information:
- Stop any currently running JBoss ON instances.
serverRoot/jon-server-3.1.0.GA/bin/rhq-server.sh stopIf the new JBoss ON server will use a database that existing JBoss ON instances are also using, then all of the existing JBoss ON instances have to be stopped.
- Download the server packages from the Customer Support Portal.
- Click Software in the top menu.
- In the Product field, select the JBoss Operations Network product.
- Unzip the server distribution to the directory where it will be executed from. For example:
cd /opt unzip jon-server-3.1.0.GA.zipThis creates a
jon-server-3.1.0.GA/directory in your chosen installation root.
- Run the JBoss ON server:
- Open the server UI at
http://localhost:7080/. This will display the JBoss ON server installer.Clicking the Click here to continue the installation link brings you to the main installer page.
- Go through the installer and fill in the information. Only the database connection information is required; most of the default can be accepted without changes.The main installer page appears different depending on the database settings.
- When the configuration is complete, click the link to log into the JBoss ON UI.
4. Fixed Bugs in JBoss Operations Network 3.1
Table 3. JBoss ON 3.1 Fixed Bugs
|Bugzilla 768707||Agent|| If the RHQ_AGENT_CMDLINE_OPTS variable was set in the |
|Bugzilla 790753||Agent|| |
The assumption is that the JBoss ON agent and resources like a JBoss server or Tomcat server run as the same system user. If the agent user is different than the resource user, then there can be problems when one entity makes a configuration change and the other attempts a change later.
For example, when deploying an application, the deployment operation is initiated by the agent and the content is supplied through the agent, and then the application server completes the actual deployment. When deleting an application, the application server handles the undeployment by itself.
If a WAR file is deployed exploded without a
|Bugzilla 787322||CLI||If the CLI were run as a service or in the background, even with a proper script file or commands passed with the CLI, the Java process would hang and the command execution failed. The CLI required that a terminal be attached to the Java process, even when it was not necessary.|
|Bugzilla 815869||Groups||In some rare cases, an autocluster (compatible) group definition could be switched to a mixed group, but the autogroup definition stayed associated with the group. Attempting to delete the dynamic mixed group (with a wrongly associated cluster group) would fail with database errors.|
|Bugzilla 819897||Groups||The number of members listed in recursive groups could be wrong because uncommitted resources listed in the discovery queue are included as implicit members.|
|Bugzilla 755703||Monitoring||The Get Live Value button for a per-minute metric for a resource returned the raw value for the metric instead of the averaged per-minute value.|
|Bugzilla 786889||Monitoring||In some rare circumstances, the agent would allow a metric schedule to be set with an invalid collection schedule, such as 0 seconds. The agent now rejects invalid metric collection schedules, automatically sets the schedule to 20 minutes, and logs an error in the agent log.|
|Bugzilla 815979||Monitoring||The memory metrics reported by JBoss ON differed from the metrics reported by native system tools because the JBoss ON calculations did not include space from caches or buffers. The memory calculation has been changed to include buffer and cache metrics, as new actual free and actual used memory, and platform utilization metrics have been updated.|
|Bugzilla 767981||Plug-in (JBoss EAP 4 and 5)|| When deploying a new ESB through the GUI, JBoss ON always installs it in the JBoss server's |
|Bugzilla 646631||Plug-in (JBoss EAP 4 and 5)||If an application is deployed to a JBoss server's farm directory and an update content operation is run, the application is no longer deployed to the farm.|
|Bugzilla 789967||Plug-in (HornetQ)||A problem in the HornetQ plug-in meant that the agent attempted to discover the parent of a HornetQ resource as a JBoss AS 5 server, even if the actual parent was a JBoss AS 4 server. This has been fixed in HornetQ 2.0.1.Final.|
|Bugzilla 817690||Plug-ins (HornetQ)||The HornetQ plug-in with JBoss ON 3.0.1 did not work with the version of HornetQ included with JBoss EAP 5. This caused Java exceptions when attempting to add HornetQ resources.|
|Bugzilla 799532||Plug-ins (JMX)||The Java attach API attempts to discover Java processes which are running as the same user as the agent. When the agent is running as root, it would sent a SIGQUIT command to attempt to connect to a Java process running as another user, but the connection attempt failed, so all the agent did was kill the other Java process. The JMX plug-in has been changed so that the agent does not attempt to connect to any Java process running as a different user.|
|Bugzilla 751873||Provisioning||If a WAR file was deployed through JBoss ON and then deleted on the local system, the JBoss server still ran the application from memory, with no attempt to undeploy the application.|
|Bugzilla 790224||Provisioning||Attempting to revert a bundle past a certain number (about 10 times) would cause the revert process to fail with a database error that the concatenated string exceeded the column width.|
|Bugzilla 804139||Provisioning||If the file content in a bundle was too small, the content manager did not properly initialize the content, and then deploying the bundle failed because the SHA256 of the initialized content did not match the stored SHA256.|
|Bugzilla 806214||Provisioning|| Uploading |
|Bugzilla 738209||Security||If an LDAP group was a member of a role, that role could not be deleted.|
|Bugzilla 747925||Server||If an agent plug-in definition was updated, the existing resources of that resource type were not updated, so new operations or metrics or other changes were not displayed for the resources.|
|Bugzilla 813083||Server||A deleted resource (such as an EAR or WAR application) is not deleted from the server inventory; it is essentially stashed. If the resource is ever re-added, then its previous audit trails and metrics histories are also preserved. However, the server improperly attempted to sync deleted resource over to the agent. When the agent then sent subsequent inventory reports to the server, it included the deleted resources — which caused the server to reject the inventory report. Now, the server inventory sync filters out the (preserved) deleted resources before sending updates to the agent.|
|Bugzilla 789529||UI||When checking the Inventory > Connection Settings tab for an autogroup, the server could throw a ClassCastException. It was checking the resource manager to see if updates were currently in progress, when it should have been checking the plug-in configuration manager.|
|Bugzilla 796096||UI||All of the bottom buttons (like Refresh) were inactivated after creating a child resource through a resource's Inventory tab.|
|Bugzilla 804303||UI||The UI did not preserve changes to the default view for a resource's Monitoring > Graphs tab. This error also prevented users from creating a custom view for a resource's monitoring graphs.|
|Bugzilla 805210||UI||When using an Oracle database as the backend, some portlets for the dashboard could fail to load because a property value could have an empty value, which Oracle interpreted as NULL.|
|Bugzilla 817165||UI||Some autogroups are automatically named after their child resource. For example, embedded WAR applications belong to an autogroup that is named for the WAR file. If the child resource changes its name, then the autogroup was renamed in the UI as "...", not the new name of the resource.|
5. Known Issues in JBoss Operations Network
5.1. Upgrade and Installation
server.example.com:7080, the browser has a cached redirect to the old UI,
server.example.com:7080/coregui. Because the
coregui/path does not exist yet, because the new server has not yet been configured, this redirect returns a 404 error.
- Enter the full URL to the installer page:
- Delete the cache and reload the page.
- Use Ctrl+F to force the page to reload.
2012-03-08 20:33:34,523 ERROR [org.rhq.enterprise.server.core.plugin.ServerPluginScanner] Failed to register server plugin file [/home/hudson/jon-server-3.0.1.GA/jbossas/server/default/deploy/rhq.ear/rhq-serverplugins/rhq-serverplugin-ant-bundle-4.2.0.JON.3.0.1.GA.jar] java.lang.NullPointerException at org.rhq.enterprise.server.core.plugin.ServerPluginScanner.registerServerPlugin(ServerPluginScanner.java:212) ...
java.rmi.server.hostnameparameter to the real value, and it uses the default of 0.0.0.0. This parameter must be set to the real IP address of the server by manually editing the
- The agent must be installed and running as a Windows service.
- No Windows Explorer window can be open to any agent subdirectory. Windows sets a lock on directories when they are open, which blocks the upgrade process.
org.apache.catalina.connector.CoyoteAdapterservice are returned when the JBoss ON 3.1 server is first installed. These errors are harmless and can be ignored. Installation will complete successfully, and both the server and the GUI will start and run properly.
5.2. JBoss EAP 6 Resources
- An edit icon appears next to module names in the Configuration > Current > Extensions area of the EAP 6 server configuration. These properties are read-only and cannot, in fact, be edited even though an edit box pops up.
- If an EAP 6 server is reset to run insecurely, then the "Install RHQ User" operation for the EAP 6 server fails because it has no way to connect to the (secure) management realm.
- Bundle provisioning is only available for EAP 6 standalone servers. Standalone servers use a filesystem-based deployment directory, which is used by the bundle system. Domain resources use a different, distributed style of deployment. The domain style of content distribution is not supported for bundles.
- When editing a network interface for a JBoss EAP 6 server, the network configuration requires either a specific IP address or a setting to use any IP address for IPv4, for IPv6, or any address. However, the UI does not enforce that an IP address selection is made, which can cause the interface to be unreachable.
- Attempting to deploy a child resource such as a WAR application fails with a malformed URL exception if the EAP 6 server is listening on IPv6.
- All child elements for an embedded HornetQ resource — connection factories, JMS queues, and JMS topics — all require a JNDI name in the configuration. Optionally, multiple JNDI names can be set, in a comma-separated list, to support failover.However, the UI does not force the JNDI name to be given when creating the connection factory, JMS queue, and JMS topic children. This means that the creation operation fails if the JNDI name is not included.
- The JBossWeb connector configuration has a section for SSL parameters. Any information in those fields is only read if SSL is enabled (the secure option is selected). However, these fields are still active even if SSL is not enabled. They should be grayed out.Also, if SSL is enabled and then disabled, all of the values for the SSL configuration fields are blanked and unset.
- When manually adding a standalone server in the JBoss ON UI, rather than importing it from the discovery queue, there is an intermittent error:
The original product type discovered for this AS7 server was null, but the server is now reporting its product type is [JBoss EAP 6]This is shown in a red bar at the top of the UI. Despite the error, the resource is successfully imported and available. This error can be ignored.
- Some path elements defined in the configuration of a domain controller or standalone server are computed by the server itself. These computed paths are all prefixed with
user.*.While the configuration values are viewable, they are not editable. However, the Configuration tab for the server shows both edit and delete options for the path elements. If a user attempts to change or delete the path, the path simply reverts to its previous state.
- A JVM definition can be created at the host, server group, and managed server levels. There can be multiple JVM definitions at the host level; these are used as templates or base definitions for the server group and managed server definitions. However, server groups and managed servers can have only a single JVM definition.The UI allows a user to attempt to create a second JVM definition as a child to a managed server or server group. If a JVM definition already exists for a managed server or server group, then an error is returned if a user attempts to create another JVM definition. The server throws an exception and there is an error message that the JVM definition is a singleton and that the first resource should be deleted before creating another child resource of that type.
- When a child JVM definition is created for a managed resource, the name of the JVM definition must be the same as the host JVM definition which it is based on. However, the JBoss ON UI creation wizard does not detect or enforce a pre-computed name, so it allows a user to enter any name.
- The UI will not let users enter a driver name when manually adding a datasource, so it is not possible to deploy a module-loaded driver.
- The agent properly detects an EAP 6 server running over an IPv6 address, but it incorrectly reports it as offline.
- It can take several minutes to be able to detect a child resource on a JBoss EAP 6 server.
- If both an EAP 6 server and a JDG server are running on the same system, the EAP 6 child resources are detected and added as (unavailable) child resources of the JDG instance.
- A user can edit configuration in JBoss ON that requires a reboot of EAP 6 to take effect, but there is no indication that the reboot is required.
5.3. Server Configuration
你好is treated as the string
你好in the installer, and the second string is the one that is hashed and stored in the
- Manually generate a hashed password based on the Unicode password:
serverRoot/jon-server-3.1.0.GA/bin/generate-db-password.sh 你好 Encoded password: 68f725778bb36d3b
- Replace the database password value with the new hashed password.
config importcommand in the agent command prompt fails with an InvalidPreferencesFormatException exception.
java.util.prefs.InvalidPreferencesFormatException: org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 41; Document root element "preferences", must match DOCTYPE root "null". ...
A fatal error has been detected by the Java Runtime Environment ...
<tag></tag>, not the abbreviated format
<rhq:system-service>Ant tag only works on Red Hat Enterprise Linux systems. Using the tag and then deploying the bundle to other operating systems causes the Ant task to fail to operate and can cause the deployment to fail.
5.8. Content and Repositories
5.9. Remote API and the CLI
Caused by: java.lang.IllegalArgumentException: interface org.rhq.enterprise.server.auth.SubjectManagerRemote is not visible from class loader at java.lang.reflect.Proxy.getProxyClass(Proxy.java:353) at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581) at org.rhq.enterprise.client.RemoteClientProxy.getProcessor(RemoteClientProxy.java:69)
coregui/element in them. Remove any old bookmarks and replace them with the new GWT-based pages in the
5.11. Measurements and Monitoring
5.14. Platform Resources
5.15. Tomcat Resources
unpackWARsoption extracts the web app to a second, invalid directory. For example, if it should be extracted to
/myapps/helloworld, the web app is extracted both there and to
5.16. Apache Resources
/tmpdirectory is configured a
fstabfile, the agent throws exceptions because it cannot properly initialize the Augeas lens. In that case, the Configuration tab is unavailable for the Apache resource.
/tmpdirectory does not have
noexecset as an option.
# # /etc/fstab # tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0
noexecis one security method available to that directory to prevent unauthorized or malicious scripts from running. Disabling
noexecfor the directory is not ideal, but it is also not a security issue by itself because other filesystem security settings can be used.
5.17. JBoss and JVM Resources
5.20. Plug-in Writing
6. Document History
|Revision 3.1-3.400||2013-10-31||Rüdiger Landmann|
|Revision 3.1-3||June 19, 2012||Ella Deon Lackey|
|Revision 3.1-1||June 12, 2012||Ella Deon Lackey|