2.11. Component Timeout Value Locations

Timeouts are useful for testing the interoperability of OpenShift components. A timeout occurs when a component sends a signal to another component, but does not receive a response. The value assigned to the timeout represents how long the component will wait for the returned signal before the process stops. All timeout values are configurable.
The following are scenarios for increasing the default timeout values:
  • When a custom cartridge is taking a long time to be added to a gear.
  • When network latency is forcing requests to take longer than usual.
  • When a high load on the system is causing actions to take longer than usual.
Note that such cases can be resolved by other methods. For example, a high load on the system can be solved by adding extra resources to the environment.
The following table outlines the locations of various component's timeout values, the configurable parameter, and the default values expressed in seconds:

Table 2.2. Timeout Information for Various Components

Type Location File Directive
MCollective Broker /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf MCOLLECTIVE_TIMEOUT=240
MCollective Node /opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.ddl :timeout => 360
MCollective Client Broker /opt/rh/ruby193/root/etc/mcollective/client.cfg plugin.activemq.heartbeat_interval = 30
Node Discovery Broker /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf MCOLLECTIVE_DISCTIMEOUT=5
Facts Broker /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf MCOLLECTIVE_FACT_TIMEOUT=10
Facts Node /opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/rpcutil.rb :timeout => 10
Apache Broker /etc/httpd/conf.d/000002_openshift_origin_broker_proxy.conf ProxyTimeout 300
Apache Node /etc/httpd/conf.d/000001_openshift_origin_node.conf ProxyTimeout 300
RHC Client ~/.openshift/express.conf timeout=300
Background Thread Broker /etc/openshift/console.conf BACKGROUND_REQUEST_TIMEOUT=30

Warning

Any modifications to the /opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.ddl and /opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/rpcutil.rb files are unsupported and may be erased by a yum update.
MCollective
The MCollective timeout is configured on the broker, and is used for MCollective messages being sent from the broker to the node. If the message is lost after it is sent, or the node takes longer than expected to complete a request, this timeout will be hit.
MCollective Client
The MCollective client timeout is used to ensure that you have a valid and active connection to your messaging broker. Lowering the defined amount causes a quicker switch to a redundant system in the event of a failure.
Node Discovery
The node discovery timeout represents the allowed amount of time a node takes to acknowledge itself in the environment, instead of broadcasting to all nodes. This method of discovery is generally used in non-direct calls to the nodes. For example, when an application is created, when some administration commands are used, and some ssh key operations are performed.
Facts
The Facts timeout is configured on both the broker and node, and is for determining the allowed amount of time for a fact to be gathered from a node through MCollective. An example of a fact is when an application is created, and in doing so, the node's profile determines which node will perform the action. Facts are gathered often, so this timeout is short.
Apache
The Apache timeout is configured on the broker and node, and represents the timeout of proxy requests. This affects most requests, as they go through a proxy on both the broker and on the node. The ProxyTimeout on the broker affects requests to the broker API and rhc. If the timeout is exceeded due to lengthy requests, the client will receive an uninformative HTTP 502 error, even though the request may have succeeded. The ProxyTimeout on a node affects requests to hosted applications.
RHC
The rhc timeout represents the allowed amount of time that the client tools will wait for a request to be completed before ceasing the attempt. This only has to be configured on the client where rhc is run. If an action is taking longer to complete than expected, this timeout will be hit.
Background Thread
The background thread timeout is found on the broker, and determines how long requests from the console to the broker will take to be completed before ceasing the attempt. This communication is impacted by the amount of applications, domains, and gears an application developer has access to, as well as the locations of the datacenters that make up the OpenShift Enterprise deployment.