3. Tuning JBoss ON for Better Performance

Every JBoss Operations Network deployment is unique, dependent on the environment which is being managed, the number and type of resources in the inventory, the configuration of individual resources, and the configuration of JBoss ON features such as metrics collection and alert definitions. These differences in environment translate into different performance for the JBoss ON server, agent, and backend database. This can make it difficult to define "universal" baselines or performance expectations.
Still (within the confines of each deployment), it is possible to identify some common behavior characteristics and potential tuning for the JBoss ON configuration which can improve server and agent performance.

NOTE

It is possible to tune a PostgreSQL or Oracle database to improve performance. That is outside the scope of the JBoss ON management. For database tuning information, see the appropriate product documentation.

3.1. Inventory Baselines

The inventory baseline was determined based on how many resources could be imported into the inventory at a single time. For an agent, this parallels first installing the agent, when all resources are discovered and imported at the same time. For the server, this parallels an upgrade operation, when a large number of existing agents are added into its inventory.

IMPORTANT

These baselines are statements of generic JBoss ON deployments in a testing environment, and the information given reflects that environment. This is intended as a reference. It is not intended as a recommendation. It may not reflect a real JBoss ON deployment which has different hardware, resources, and other factors.
Server Baselines
A server could typically upgrade with 100 or more agents before encountering out of memory errors. If those errors are encountered, the server settings can be tuned as described in Section 3.6, “Server Tuning for Large Numbers of Agents” to increase the thread pool and concurrency limits to allow more requests to be processed.
Agent Import Baselines
The resources on a system can be arranged in two slightly different ways. One way is a flat hierarchy, where there are fewer levels in the hierarchy and each level is very broad. Essentially, this is few parent resources with lots of child resources.
                                           platform
                                              |
                          ------------------------------------------------
                          |         |         |         |         |      |
                       server1   server2   server3   server4   server5   ...
                          |
   -----------------------------------------------------
   |           |          |          |          |      |   
service1   service2   service3   service4   service5   ...
Alternatively, the hierarchy can be very deep, where parent resources have few direct children but there are levels of child services. This is especially common with EAP 6 and similar applications which have many subsystems and services.
         platform
            |
  ---------------------
  |         |         |
server1   server2   server3
  |
  -------------
  |          |
service1   service2
   |
   ------------
   |          |
serviceA   serviceB
   |
   ------------
   |          |
serviceI   serviceII
   |
   ------------
   |          |
serviceX   serviceY
   |
   ---
   |          
service...
The hierarchy of the resources does impact how the agent performs when importing large inventories because of how it recurses through the parent and child resources.

Table 7. Agent Import Performance

Resource Hierarchy Tested Import Time Notes
Deep hierarchy: 10 top-level servers, 10 mid-level servers, 750 child services (10x10x750)
(75,000 total resources)
2 hours 46 minutes Increasing the agent heap size to 2GB moderately decreased the import time, to 2 hours 34 minutes. It also reduced the risk of out-of-memory errors, which could occur when importing a large number of resources with the normal agent memory settings.
Flat hierarchy: 100 top-level servers, 750 child services (10x10x750)
(75,000 total resources)
2 hours 14 minutes

After garbage collection, the memory footprint during the import operation was an average of 665.3MB. After the import completed, the memory footprint was 586.2MB.
Agent and EAP 6 Resource Baselines
A single agent can manage 40 to 50 JBoss Enterprise Application Platform 6 instances.

NOTE

The number of EAP 6 resources which can be managed by a single agent is highly dependent on the hardware of the underlying system, the number of web contexts managed by each EAP instance, the size of the web contexts, and other factors.