6. Example: Managing Grouped Servers (JBoss EAP 5)

A lot of enterprise servers have a concept of managed servers. A managed server means that there is a central instance that deploys content or sends configuration to all registered application servers. Using managed servers helps administrators ensure that all active application servers have the same version of the deployed packages and configuration.
JBoss ON can imitate the behavior of managed or clustered servers for applications like Tomcat or JBoss EAP 5 by creating a management script that can be invoked to perform actions simultaneously on all members of a JBoss ON group. All of the EAP 5 instances are functionally managed servers, while JBoss ON itself acts as the domain controller.

NOTE

JBoss EAP 6 has a very different server topology than JBoss EAP 5, so domain controllers, managed servers, and domain configuration are defined and manageable by default.

6.1. The Plan for the Scripts

The JBoss ON CLI can run defined JavaScripts using the -f parameter. The idea here is to create a series of small management scripts that perform specific tasks on a group of JBoss EAP servers. This example has seven scripts for:
  • Creating a group
  • Adding EAP instances to the group
  • Checking EAP status
  • Starting the EAP instance
  • Scheduling an operation
  • Deploying new content to the group
  • Checking metrics
A wrapper script and configuration file will be set up so that only one command needs to be run; the wrapper invokes the appropriate JBoss ON CLI script depending on the command passed to the wrapper.

6.2. Creating the Wrapper Script and .conf File

The wrapper script takes command-line arguments and calls the JBoss ON CLI with one of the scripts as argument. The command-line arguments themselves are defined in the JBoss ON JavaScript files.
This wrapper script makes a few assumptions:
  • The wrapper script is run as a regular user, which means that any JavaScript files must be accessible to a regular user.
  • The scripts are located in a scripts/ directory that is in the same directory as the wrapper script.
  • A separate configuration file defines connection information for the JBoss ON server.
  • Each JavaScript file is invoked by a separate CLI command invocation, defined in the wrapper.
  • Any options or information required by the JBoss ON CLI command is defined in the JavaScript file and can, potentially, be passed with the wrapper script as an option.
#!/bin/bash
#
# groupcontrol
# ------------
# This is a simple wrapper script for all the java script scripts in this folder.
# Start this script with some parameters to automate group handling from within the
# command line.
# 
# With groupcontrol you can do the following:
#   create   : Create a new group
#   addMember: Add a new EAP instance to the specified group
#   status   : Print the status of all resources of a group
#   start    : start all EAP instances specified by group name
#   deploy   : Deploys an application to all AS instances specified by group name
#   avail    : Runs an availability operation on all discovered agent instances
#   metrics  : Gets the specified metric value for all AS instances specified by group name
#
# 
			
## Should not be run as root.
if [ "$EUID" = "0" ]; then
   echo " Please use a normal user account and not the root account"
   exit 1
fi
		
## Figure out script home
MY_HOME=$(cd `dirname $0` && pwd)
SCRIPT_HOME=$MY_HOME/scripts
			
## Source some defaults
. $MY_HOME/groupcontrol.conf
			
## Check to see if we have a valid CLI home
if [ ! -d ${JON_CLI_HOME} ]; then
     echo "JON_CLI_HOME not correctly set. Please do so in the file"
     echo $MY_HOME/groupcontrol.conf
     exit 1
fi
			
RHQ_OPTS="-s $JON_HOST -u $JON_USER -t $JON_PORT"
# If JBoss ON_PWD is given then use it as argument. Else let the user enter the password
if [ "x$JON_PWD" == "x" ]; then
     RHQ_OPTS="$RHQ_OPTS -P"
else
     RHQ_OPTS="$RHQ_OPTS -p $JON_PWD"
fi
			
#echo "Calling groupcontrol with $RHQ_OPTS"
			
usage() {
     echo "  Usage $0:"
     echo "  Use this tool to control most group related tasks with a simple script."
     echo "  ------------------------------------------------------------------------- "
}
Each command that the wrapper should define has a doCommand() section which defines the JBoss ON CLI command to run and the JavaScript file to use.
doDeploy() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/deploy.js $2 $3
}

doCreate() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/group.js $2
}
					
doAddMember() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/addMember.js $2 $3 $4
}	
			
doStatus() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/status.js $2
}

doRestart() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/restart.js $2 
}

doAvail() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/avail.js
}	
		
doMetrics() {
     $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/metrics.js $2 $3
}

case "$1" in
'deploy')
	doDeploy $*
	;;     
'create')
	doCreate $*
	;;     
'addMember')
	doAddMember $*
	;;     
'status')
	doStatus $*
	;;     
'restart')
	doRestart $*
	;;     
'avail')
	doAvail $*
	;;     
'metrics')
	doMetrics $*
	;;     
*)
        usage $*
        ;;
esac
This script uses a configuration file, groupcontrol.conf, which defines the connection information to connect to the JBoss ON server (which is required by the JBoss ON CLI).
##	
## This file contains some defaults for the groupcontrol script
##
JON_CLI_HOME=cliRoot/rhq-remoting-cli-3.1.2.GA
JON_HOST=localhost
JON_PORT=7080
			
# The user you want to connect with
JON_USER=rhqadmin
			
# if you omit the password here, you'll be prompted for it.
JON_PWD=rhqadmin

6.3. Defining Arguments and Other Parameters for the CLI Scripts

There may be multiple groups or some tasks (like searching for resources or running an operation) may have multiple options.
Each JavaScript file can define its own script options in args methods. At a minimum, each script should accept the name of the group on which to perform the task.
It is also a really good idea to define a usage function, so that each command can print what options are expected. For example:
function usage() {
        println("Usage: deploy groupName");
        throw "Illegal arguments";
}

if( args.length < 1 ) usage();
var groupName = args[0];

NOTE

When adding arguments for a script, be sure to set the proper number of tokens in the wrapper script for the CLI invocation. For example, for groupName and fileName, add $2 $3.
doDeploy() {
      $JON_CLI_HOME/bin/rhq-cli.sh $RHQ_OPTS -f $SCRIPT_HOME/deploy.js $2 $3
}
Aside from the script for creating a group, every script must also include a search for the group to perform the operations on. For example:
groupcriteria = new ResourceGroupCriteria();
groupcriteria.addFilterName(groupName);

var groups = ResourceGroupManager.findResourceGroupsByCriteria(groupcriteria);
if( groups != null ) {
  if( groups.size() > 1 ) {
        println("Found more than one group.");
  }
  else if( groups.size() == 1 ) {
     group = groups.get(0);
  }
}

6.4. Creating a Group: group.js

Set up the script. This script only uses a single argument, for the name of the new group (groupName). The resource type in the example is hard-coded to JBossAS5, which is a JBoss AS 5 server; optionally, it is possible to also add arguments to set the plug-in name and type so that other JBoss versions could be specified.
function usage() {
        println("Usage: deploy groupName");
        throw "Illegal arguments";
}

if( args.length < 1 ) usage();
var groupName = args[0];
Create the group:
var rg = new ResourceGroup(resType);
rg.setRecursive(false);
rg.setDescription("Created via groupcontrol scripts on " + new java.util.Date().toString());
rg.setName(groupName);

rg = ResourceGroupManager.createResourceGroup(rg);

var resType = ResourceTypeManager.getResourceTypeByNameAndPlugin("JBossAS 5 Server","JBossAS5");

6.5. Adding Resources to a Group: addMember.js

Set up the script. This identifies three required arguments for the script:
  • groupName for the group to add the resources to
  • resourceName for the name of the resource to add; this is one of the search criteria
  • resourceTypeName for the type of resource to add; this is one of the search criteria
This also includes a search to find the group specified in the argument.
function usage() {
        println("Usage: addMember groupName resourceName resourceTypeName");
        throw "Illegal arguments";
}

if( args.length < 3 ) usage();
var groupName = args[0];
var resourceName = args[1];
var resourceTypeName = args[2];

groupcriteria = new ResourceGroupCriteria();
groupcriteria.addFilterName(groupName);

var groups = ResourceGroupManager.findResourceGroupsByCriteria(groupcriteria);
if( groups != null ) {
  if( groups.size() > 1 ) {
        println("Found more than one group.");
  }
  else if( groups.size() == 1 ) {
     group = groups.get(0);
  }
}
Search for the resources to add to the group. The script is designed to add only a single resource to the group, so the given search criteria, resourceName and resourceTypeName, must be specific enough to match only a single resource.
criteria = new ResourceCriteria();
criteria.addFilterName(resourceName);
criteria.addFilterResourceTypeName(resourceTypeName);

var resources = ResourceManager.findResourcesByCriteria(criteria);
if( resources != null ) {
  if( resources.size() > 1 ) {
        println("Found more than one JBossAS Server instance. Try to specialize.");
     for( i =0; i < resources.size(); ++i) {
          var resource = resources.get(i);
          println("  found " + resource.name );
     }
  }
  else if( resources.size() == 1 ) {
     resource = resources.get(0);
     println("Found one JBossAS Server instance. Trying to add it.");
     println("  " + resource.name );
        ResourceGroupManager.addResourcesToGroup(group.id, [resource.id]);
     println("  Added to " + group.name + "!");
  }
  else {
        println("Did not find any JBossAS Server instance matching your pattern. Try again.");
  }
}
When this script is run, it prints the name of the found JBoss instance and that it was added to the group.
[jsmith@server cli]$ ./wrapper.sh addMember myGroup "JBossAS App 1" "JBossAS Server"
Remote server version is: 3.0.1.GA (b2cb23b:859b914)
Login successful
Found one JBossAS Server instance. Trying to add it.
  AS server.example.com JBossAS App 1
  Added to myGroup!

6.6. Getting Inventory and Status Information: status.js

This is a simple little script, just to print the current status of all the JBoss instances in the group.
As with the other scripts, set up the group information.
function usage() {
        println("Usage: status groupName");
        throw "Illegal arguments";
}

if( args.length < 1 ) usage();
var groupName = args[0];

groupcriteria = new ResourceGroupCriteria();
groupcriteria.addFilterName(groupName);

var groups = ResourceGroupManager.findResourceGroupsByCriteria(groupcriteria);
if( groups != null ) {
  if( groups.size() > 1 ) {
        println("Found more than one group.");
  }
  else if( groups.size() == 1 ) {
     group = groups.get(0);
  }
}
Also include information to search for the resources, based on the group:
criteria = new ResourceCriteria();
criteria.addFilterExplicitGroupIds(group.id);

var resources = ResourceManager.findResourcesByCriteria(criteria);
for( i =0; i < resources.size(); ++i) {
     var resource = resources.get(i);
     println("  found " + resource.name );
}
Then, run through the resources and print their availability.
var server = ProxyFactory.getResource(resource.id);
var avail  = AvailabilityManager.getCurrentAvailabilityForResource(server.id);

println("  " + server.name );
println("    - Availability: " + avail.availabilityType.getName());
println("    - Started     : " + avail.startTime.toGMTString());
println("");

var avail = AvailabilityManager.getCurrentAvailabilityForResource(server.id);

if( avail.availabilityType.toString() == "DOWN" ) {
           println("  Server is DOWN. Please first start the server and run this script again!");
           println("");
}
When the script is run, it prints the availability and last start time for the servers.
[jsmith@server cli]$ ./wrapper.sh status myGroup
Remote server version is: 3.0.1.GA (b2cb23b:859b914)
Login successful
  found AS server.example.com JBossAS App 1
  AS server.example.com JBossAS App 1
    - Availability: UP
    - Started     : 11 Feb 2012 04:07:37 GMT

6.7. Starting, Stopping, and Restarting the Server: restart.js

Set up the script with the usage information and the group search, as in Section 6.6, “Getting Inventory and Status Information: status.js”.
This example only performs one operation, restarting a JBoss server. It iterates through all the resources in the group.
It is possible to write similar scripts for starting and stopping the server.
  • shutdown() for AS4 servers and shutDown() for AS5 servers
  • start()
criteria = new ResourceCriteria();
criteria.addFilterExplicitGroupIds(group.id);

var resources = ResourceManager.findResourcesByCriteria(criteria);
for( i =0; i < resources.size(); ++i) {
     var resource = resources.get(i);
     var resType = resource.resourceType.name;
     println("  found " + resource.name );

     if( resType != "JBossAS Server") {
          println("    ---> Resource not of required type. Exiting!");
          usage();
     }

     var server = ProxyFactory.getResource(resource.id);
     println("    stopping " + server.name + "....");
     try {
         server.shutdown()
     }
     catch( ex ) {
         println("   --> Caught " + ex );
     }
				
     println("    restarting " + server.name + "....." );
     try {
         server.start();
     }
     catch( ex ) {
         println("   --> Caught " + ex );
     }
}

6.8. Deploying Applications to the Group Members: deploy.js

Set up the usage information and the group search as in the other scripts, then use the deployment script described in Section 3, “Example: Scripting Resource Deployments (JBoss EAP 5)”.
The script uses two parameters, one for the group name and one for the file to upload.
As one easy improvement, the last part of Section 3.2, “Checking the JBoss ON Groups and Inventory” stops the JBoss server, uploads the content, and restarts it. Instead, simply check that the server is running first, and then upload the content:
// we need check to see if the given server is up and running
var avail = AvailabilityManager.getCurrentAvailabilityForResource(server.id);
				
// unfortunately, we can only proceed with deployment if the server is running. Why?
if( avail.availabilityType.toString() == "DOWN" ) {
	   println("  Server is DOWN. Please first start the server and run this script again!");
	   println("");
	   continue;
}

6.9. Scheduling an Availability Operation: avail.js

Unlike the other tasks in this script set, the operation task is run on the agent, so it is not necessary to search for the group or JBoss resource. This runs an availability scan on the agent; it is also possible to run a specific command on the agent using the Execute prompt command operation.
First, get a list of all agent resources:
println("Scanning all RHQ Agent instances");
var rc = ResourceCriteria();
var resType = ResourceTypeManager.getResourceTypeByNameAndPlugin("RHQ Agent", "RHQAgent");
rc.addFilterPluginName("RHQAgent");
rc.addFilterResourceTypeName("RHQ Agent");
rc.addFilterParentResourceTypeId("10001");

var resources = ResourceManager.findResourcesByCriteria(rc).toArray();

var idx=0;
for( i in resources ) {
     if( resources[i].resourceType.id == resType.id ) {
          resources[idx] = resources[i];
          idx = idx + 1;
     }
}
Then, traverse the agents array and schedule the operation:
for( a in resources ) {
     var agent = resources[a]

     var resType = agent.resourceType.name;
     println("  Found resource " + agent.name + " of type " + resType + " and ID " + agent.id);

     println("  executing availability scan on agent" );
     println("    -> " + agent.name + " / " + agent.id);
     var config = new Configuration();
     config.put(new PropertySimple("changesOnly", "true") );
     var ros = OperationManager.scheduleResourceOperation(
          agent.id,
          "executeAvailabilityScan",
          0,
          1,
          0,
          10000000,
          config,
          "test from cli"
     );

     println(ros);
     println("");
}

6.10. Gathering Metric Data of Managed Servers: metrics.js

JBoss ON collects a number of metrics for each resource type. This information can be retrieved by using the findLiveData method, which returns the current active value for the resource.
This script takes two arguments, the groupName and the metricName. As with the other scripts, this searches for the group and then the resource by the group ID.
function usage() {
        println("Usage: metrics groupName metricName");
        throw "Illegal arguments";
}

if( args.length < 2 ) usage();
var groupName = args[0];
var metricName = args[1];

groupcriteria = new ResourceGroupCriteria();
groupcriteria.addFilterName(groupName);

var groups = ResourceGroupManager.findResourceGroupsByCriteria(groupcriteria);
if( groups != null ) {
  if( groups.size() > 1 ) {
        println("Found more than one group.");
  }
  else if( groups.size() == 1 ) {
     group = groups.get(0);
  }
}

criteria = new ResourceCriteria();
criteria.addFilterExplicitGroupIds(group.id);
The actual metric search looks for the metrics available to the resource type (hard-coded to JBoss AS 5 in this example). The metric itself is identified solely by the metricName argument.
var rt = ResourceTypeManager.getResourceTypeByNameAndPlugin("JBossAS 5 Server","JBossAS5");
var mdc = MeasurementDefinitionCriteria();
mdc.addFilterDisplayName(metricName);
mdc.addFilterResourceTypeId(rt.id);
var mdefs =  MeasurementDefinitionManager.findMeasurementDefinitionsByCriteria(mdc);
var resources = ResourceManager.findResourcesByCriteria(criteria);
var metrics = MeasurementDataManager.findLiveData(resources.get(0).id, [mdefs.get(0).id]);

if( metrics !=null ) {
        println(" Metric value for " + resources.get(0).id + " is " + metrics );
}
When the script is run, it prints the resource ID and the current value for the metric.
[jsmith@server cli]$ ./wrapper.sh metrics myGroup "Active Thread Count"
Remote server version is: 3.0.1.GA (b2cb23b:859b914)
Login successful
 Metric value for 10003 is [MeasurementDataNumeric[value=[64.0], MeasurementData [MeasurementDataPK: timestamp=[Wed Feb 15 22:14:38 EST 2012], scheduleId=[1]]]]