Red Hat Training

A Red Hat training course is available for Red Hat JBoss Operations Network

9.9. Adjusting the Capacity of the Storage Node to Receive Metrics (Storage Node Request Throttling)

The JBoss ON storage node automatically throttles the request capacity (incoming metrics capacity), however in some situations manual tuning can be required. Manual tuning can be performed by adjusting the Storage Client Settings within the rhq-server.properties file or the JBoss ON GUI.

9.9.1. Storage Node Request Limit Settings (Storage Client Settings)

rhq.storage.request.limit.topology-delta=30000
The topology delta defines the request limit increase when additional storage nodes are added, expressed as requests per minute. For example, assuming the existing storage node has a limit of 30,000 requests per minute, adding a new storage node adds 30,000 requests per minute, increasing the limit to 60,000 requests per minute. For more details on adding additional storage nodes, see Section 9.2.4, “Installing Additional Nodes”. 
rhq.storage.request.limit.timeout-dampening=30000
The timeout dampener defines the period of time where successive timeouts are ignored, expressed in milliseconds (default 30,000ms). This timeout dampening property assumes that when a timeout occurs, there may be timeouts that immediately follow. 
rhq.storage.request.limit.timeout-delta=0.2
(Deprecated from JBoss ON version 3.3.1) The timeout delta defines the proportional decrease (between 0 - 1, with a default of 0.2) in the request limit when a request timeout occurs. This assumes that request timeouts are caused by the hardware of the storage node having insufficient capacity to handle requests at the current request limit. For example, if the initial limit is 30,000 requests per minute and a request timeout occurs, the limit decreases by 20% to 24,000 requests per minute (0.2 * 30,000 requests per minute = 6,000 decrease in request limit, therefore the new request limit is 24,000 requests). 
rhq.storage.request.limit.min=5000
(Deprecated from JBoss ON version 3.3.1) The rhq.storage.request.limit.min sets the minimum request limit. The storage request limit can not decrease below this value regardless of the number of timeouts. 
rhq.storage.request.limit.warmup-period=3
(Introduced by the JBoss ON 3.3.1 update) The warm up period sets the increase in warm up time when storage request timeouts occur (expressed in minutes). 
rhq.storage.request.limit.max-warmup-counter=10
(Introduced by the JBoss ON 3.3.1 update) The maximum warm up counter sets the maximum warm up time after multiple successive storage request timeouts. Maximum warm up time is a function of the warm up counter and the warm up period. For example, warmup-period=3 multiplied by max-warmup-counter=10 gives a maximum warm up time of 30 minutes.
Using storage request limit warm up settings
If a timeout occurs, the storage request limit is halved. The storage request limit returns to 100% of the configured storage request limit over the warm up time, initially equal to rhq.storage.request.limit.warmup-period. If subsequent timeouts occur during the warm up time (after accounting for the rhq.storage.request.limit.timeout-dampening), the warm up time increases by the warm up period up to a maximum warm up time. The maximum warm up time is given by: rhq.storage.request.limit.max-warmup-counter multiplied by rhq.storage.request.limit.warmup-period (the default maximum warm up time is 30 minutes).

9.9.2. Manually Change the Storage Node Request Limit Settings in rhq-server.properties

The storage node request limit settings can be changed manually in <JBoss_ON_install_directory>/bin/rhq-server.properties under the section Storage Client Settings.
The warm up settings for RHQ storage request limits and <JBoss_ON_install_directory>/bin/rhq-server.properties.new were added in JBoss ON version 3.3.1 . To change the storage request limit warm up settings (warmup-period and max-warmup-counter) within the <JBoss_ON_install_directory>/bin/rhq-server.properties file, the settings need to be manually added as presented in Section 9.9.1, “Storage Node Request Limit Settings (Storage Client Settings)” or can be copied from <JBoss_ON_install_directory>/bin/rhq-server.properties.new.

9.9.3. Using the JBoss ON GUI to Adjust Storage Node Request Throttling

To change the Storage Node Request Throttling settings within the JBoss ON GUI:
  1. Select the Inventory tab.
  2. From the Resources menu, select All Resources.
  3. Search for the Storage Client Subsystem and select the name to open.
  4. From the Storage Client Subsystem page, select the Configuration tab.
  5. Make your changes to the properties. Select the Unset? check box to use the default value, or clear the Unset? check box to use a custom value.
  6. Click Save at the bottom of the page.

Figure 9.4. Example of Storage Node Request Throttling settings within the JBoss ON GUI.

Example of Storage Node Request Throttling settings within the JBoss ON GUI.
Report a bug