Red Hat Training

A Red Hat training course is available for Red Hat JBoss Operations Network

9.2. Deploying and Undeploying Storage Nodes

There can be multiple metrics storage nodes in the JBoss ON environment. Much like the JBoss ON servers can operate in a high availability cloud, multiple storage nodes are deployed in the same environment and communicate with each other to function in a cluster.
Nodes can be added and removed from the cluster by administrators or dynamically using JBoss ON scripts in response to changes in the environment.
Note
A companion agent is always installed with a storage node to enable server-node communication.
With the default cluster configuration, a node is deployed as soon as it is installed and configured in the JBoss ON server. These are actually two separate steps.
  1. The bits are installed on a local system and the storage node is registered with the JBoss ON server.
  2. The new node information is deployed to the cluster.
Whether to deploy nodes automatically or manually can be determined by the provisioning and monitoring requirements of the environment.
Warning
Deploying a node lists that node's host in the cluster configuration and any allowed host can gain access to the data in the storage cluster.
Restrict access to the rhq-storage-auth.conf file so that the allowed hosts list cannot be altered to allow an attacker to gain access to the cluster and the stored data.
Report a bug

9.2.1. Storage Node Requirements

There are two critical requirements for deploying nodes:
  • The hostnames or IP addresses of all storage nodes and the hostname and bind addresses of the JBoss ON server and agents must all be fully-resolvable in DNS. If the IP addresses and hostnames of the storage nodes, servers, or agents are not properly formatted in DNS, then all communication between the different JBoss ON components will fail.
  • The firewall must allow communication over the two ports used by the storage nodes. By default, the ports are 9142 and 7100 for the server/client and gossip ports, respectively.
Report a bug

9.2.2. Installing the Storage Node with the Server

By default, the server installation script also installs the storage node and an agent: 
[root@server ~]# serverRoot/jon-server-3.3.0.GA/bin/rhqctl install --start
No additional options are required.
Report a bug

9.2.3. Installing Storage Nodes Before Installing the Server

It is possible to create multiple storage nodes before installing a server, and then install the server with those pre-installed nodes. This is also useful if the storage database will be on a separate, dedicated machine.
Warning
This is an advanced configuration. If the storage node or nodes within the cluster are not properly configured, then the cluster may not properly function.
Warning
Deploying a node lists that node's host in the cluster configuration and any allowed host can gain access to the data in the storage cluster.
Restrict access to the rhq-storage-auth.conf file so that the allowed hosts list cannot be altered to allow an attacker to gain access to the cluster and the stored data.
Important
Every storage node must use the same client (CQL) and gossip ports.
Additionally, the hostname and IP address of every storage node system must be fully resolvable in DNS or must be configured on each system's hosts file.
  1. Determine the node and cluster configuration information to use.
    • Identify the hostname or IP address of each system which will host a node.
    • Define the two ports which the cluster uses for communication (9142 and 7100 by default).
  2. Before installing any storage node, edit the storage properties file with all of the node and cluster information. 
    [root@server ~]# vim serverRoot/jon-server-3.3.0.GA/bin/rhq-storage.properties
    For example, this configures three nodes, set in the rhq.storage.seeds parameter. 
    rhq.storage.cql-port=9142
rhq.storage.gossip-port=7100
rhq.storage.seeds=192.68.0.0, 192.68.0.1, 192.68.0.2
start=false
  3. Install the storage node on each system, with its companion agent. This requires the IP address of the JBoss ON server, even though the server is not yet installed.
    Do not start the storage node or the agent at this point. Do not use the --start option with the installation script. 
    [root@server ~]# serverRoot/jon-server-3.3.0.GA/bin/rhqctl install --storage --agent-preference="rhq.agent.server.bind-address=192.68.0.2"
  4. For each storage node, edit its local rhq-storage-auth.conf file. This lists the hostnames or IP addresses for all of the storage nodes in the cluster, one per line. 
    [root@server ~]# vim serverRoot/jon-server-3.3.0.GA/rhq-storage/conf/rhq-storage-auth.conf 

192.68.0.0
192.68.0.1
192.68.0.2
    After the server is configured, the local agent will update the rhq-storage-auth.conf file with node hostnames or IP addresses as nodes are deployed and removed from the cluster.
  5. Start each node. 
    [root@server ~]# serverRoot/jon-server-3.3.0.GA/bin/rhqctl start --storage
  6. Before installing the server, edit the rhq-server.properties file to include the connection information for the storage nodes.
    Add each storage node in a comma-separated listed to the rhq.storage.nodes parameter. Then, add the client and gossip port values. 
    [root@server ~]# vim serverRoot/jon-server-3.3.0.GA/bin/rhq-server.properties

rhq.storage.nodes=192.68.0.0,192.68.0.1,192.68.0.2
rhq.storage.cql-port=9142
rhq.storage.gossip-port=7100
  7. Install the server and an agent. Specifying the --server and --agent options only installs those two components; the storage database is excluded. 
    [root@server ~]# serverRoot/jon-server-3.3.0.GA/bin/rhqctl install --server --agent --start
    If you are upgrading an existing JBoss ON agent, then run the upgrade script with the --use-remote-storage-note option, to load the storage database information from the properties file rather than installing a storage node. 
    [root@server]# serverRoot/jon-server-3.3.0.GA/bin/rhqctl upgrade --use-remote-storage-node=true
Report a bug

9.2.4. Installing Additional Nodes

When creating a new storage node, two components are always installed: the storage node and a companion agent. The only required information to set up the node is the IP address of the JBoss ON server. The agent registers with the server and then sends the storage node hostname or IP address. The server then distributes that information among the other agents and it gets propagated into the cluster.
If the cluster is using custom client and gossip ports, then edit the rhq-storage.properties file with the correct ports before running the installation script.
Run the installation script with the --agent-preference option to supply the server bind address. For example: 
[root@server ~]# serverRoot/jon-server-3.3.0.GA/bin/rhqctl install --storage --agent-preference="rhq.agent.server.bind-address=0.0.0.0"
Note
When installing on Linux, the rhqctl command must be run as root. On Windows, the command prompt must be opened with the option Run as Administrator.
The deployment operation can take several minutes to complete, as the new node information is propagated among the existing nodes. Until the deploy operation is complete, the node shows a status of Joining.

Figure 9.2. Joining the Cluster

Joining the Cluster
Warning
Deploying a node lists that node's host in the cluster configuration and any allowed host can gain access to the data in the storage cluster.
Restrict access to the rhq-storage-auth.conf file so that the allowed hosts list cannot be altered to allow an attacker to gain access to the cluster and the stored data.
Report a bug

9.2.5. Setting Automatic Deployment for Nodes

When a node is created with the proper port and cluster settings, it can either join the cluster immediately or it can wait, essentially on standby, and then be deployed manually later. This is determined by a cluster-wide setting.
This can be useful if machines are being provisioned and taken offline repeatedly (such as in virtual environments) or if nodes should only be online during certain periods.
  1. Click the Administration tab in the top navigation bar.
  2. In the Topology area on the left, select the Storage Nodes item.
  3. Open the Cluster Settings tab.
  4. The Automatic Deployment option sets whether to deploy a storage node to the cluster as soon as it is installed in JBoss ON or to wait to deploy it manually.
    The default is to deploy nodes automatically. This can be changed if the environment has other provisioning rules in place.
  5. Click Save at the bottom of the page.
Report a bug

9.2.6. Deploying Nodes Manually

By default, when a new storage node is installed, it is automatically deployed to the cluster. However, if automatic deployment is disabled, then new nodes must be deployed manually after they are created.
Warning
Deploying a node lists that node's host in the cluster configuration and any allowed host can gain access to the data in the storage cluster.
Restrict access to the rhq-storage-auth.conf file so that the allowed hosts list cannot be altered to allow an attacker to gain access to the cluster and the stored data.
  1. Install a node using the rhqctl install command.
  2. Click the Administration tab in the top navigation bar.
  3. In the Topology area on the left, select the Storage Nodes item.
  4. In the Nodes tab, select the row of the node to deploy.
  5. Click the Deploy button.
A node can also be deployed by running a Deploy operation on the storage node resource.
Note
Storage nodes can be deployed using the JBoss ON CLI and scripts, as well. This can be useful when provisioning new systems in the infrastructure.
For example: 
// deploy a storage node 
nodes = StorageNodeManager.findStorageNodesByCriteria(StorageNodeCriteria()); 
node = nodes.get(0); StorageNodeManager.deployStorageNode(node);
This can be key in dynamically adding storage nodes as demands on the infrastructure increase. Nodes can be installed in advance and then hot-deployed and removed as necessary.
Report a bug

9.2.7. Removing Nodes

Undeploying a storage node removes it from the cluster and then removes the resource from the inventory and uninstalls the storage node bits from the machine.
  1. Click the Administration tab in the top navigation bar.
  2. In the Topology area on the left, select the Storage Nodes item.
  3. In the Nodes tab, select the row of the node to remove. To select multiple rows, hold the Ctrl key and click the desired rows.
  4. Click the Undeploy Selected button, and confirm the operation.
A node can also be removed by running an Undeploy operation on the storage node resource.
Note
Storage nodes can be removed using the JBoss ON CLI and scripts, as well. This can be useful when provisioning and removing systems from the infrastructure.
For example: 
// undeploy a storage node 
nodes = StorageNodeManager.findStorageNodesByCriteria(StorageNodeCriteria()); 
node = nodes.get(0); StorageNodeManager.undeployStorageNode(node);
Report a bug