Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

Chapter 6. Data processing service

The Data processing service (sahara) provides a scalable data-processing stack and associated management interfaces.
The following tables provide a comprehensive list of the Data processing service configuration options.

Table 6.1. Description of AMQP configuration options

Configuration option = Default value Description
[DEFAULT]
control_exchange = openstack (StrOpt) The default exchange under which topics are scoped. May be overridden by an exchange name specified in the transport_url option.
notification_driver = [] (MultiStrOpt) The Drivers(s) to handle sending notifications. Possible values are messaging, messagingv2, routing, log, test, noop
notification_level = INFO (StrOpt) Notification level for outgoing notifications
notification_publisher_id = None (StrOpt) Notification publisher_id for outgoing notifications
notification_topics = notifications (ListOpt) AMQP topic used for OpenStack notifications.
transport_url = None (StrOpt) A URL representing the messaging driver to use and its full configuration. If not set, we fall back to the rpc_backend option and driver specific configuration.

Table 6.2. Description of API configuration options

Configuration option = Default value Description
[oslo_middleware]
max_request_body_size = 114688 (IntOpt) The maximum body size for each request, in bytes.
secure_proxy_ssl_header = X-Forwarded-Proto (StrOpt) The HTTP Header that will be used to determine what the original request protocol scheme was, even if it was hidden by an SSL termination proxy.
[retries]
retries_number = 5 (IntOpt) Number of times to retry the request to client before failing
retry_after = 10 (IntOpt) Time between the retries to client (in seconds).

Table 6.3. Description of authorization token configuration options

Configuration option = Default value Description
[keystone_authtoken]
admin_password = None (StrOpt) Service user password.
admin_tenant_name = admin (StrOpt) Service tenant name.
admin_token = None (StrOpt) This option is deprecated and may be removed in a future release. Single shared secret with the Keystone configuration used for bootstrapping a Keystone installation, or otherwise bypassing the normal authentication process. This option should not be used, use `admin_user` and `admin_password` instead.
admin_user = None (StrOpt) Service username.
auth_admin_prefix = (StrOpt) Prefix to prepend at the beginning of the path. Deprecated, use identity_uri.
auth_host = 127.0.0.1 (StrOpt) Host providing the admin Identity API endpoint. Deprecated, use identity_uri.
auth_plugin = None (StrOpt) Name of the plugin to load
auth_port = 35357 (IntOpt) Port of the admin Identity API endpoint. Deprecated, use identity_uri.
auth_protocol = https (StrOpt) Protocol of the admin Identity API endpoint (http or https). Deprecated, use identity_uri.
auth_section = None (StrOpt) Config Section from which to load plugin specific options
auth_uri = None (StrOpt) Complete public Identity API endpoint.
auth_version = None (StrOpt) API version of the admin Identity API endpoint.
cache = None (StrOpt) Env key for the swift cache.
cafile = None (StrOpt) A PEM encoded Certificate Authority to use when verifying HTTPs connections. Defaults to system CAs.
certfile = None (StrOpt) Required if identity server requires client certificate
check_revocations_for_cached = False (BoolOpt) If true, the revocation list will be checked for cached tokens. This requires that PKI tokens are configured on the identity server.
delay_auth_decision = False (BoolOpt) Do not handle authorization requests within the middleware, but delegate the authorization decision to downstream WSGI components.
enforce_token_bind = permissive (StrOpt) Used to control the use and type of token binding. Can be set to: "disabled" to not check token binding. "permissive" (default) to validate binding information if the bind type is of a form known to the server and ignore it if not. "strict" like "permissive" but if the bind type is unknown the token will be rejected. "required" any form of token binding is needed to be allowed. Finally the name of a binding method that must be present in tokens.
hash_algorithms = md5 (ListOpt) Hash algorithms to use for hashing PKI tokens. This may be a single algorithm or multiple. The algorithms are those supported by Python standard hashlib.new(). The hashes will be tried in the order given, so put the preferred one first for performance. The result of the first hash will be stored in the cache. This will typically be set to multiple values only while migrating from a less secure algorithm to a more secure one. Once all the old tokens are expired this option should be set to a single value for better performance.
http_connect_timeout = None (IntOpt) Request timeout value for communicating with Identity API server.
http_request_max_retries = 3 (IntOpt) How many times are we trying to reconnect when communicating with Identity API Server.
identity_uri = None (StrOpt) Complete admin Identity API endpoint. This should specify the unversioned root endpoint e.g. https://localhost:35357/
include_service_catalog = True (BoolOpt) (Optional) Indicate whether to set the X-Service-Catalog header. If False, middleware will not ask for service catalog on token validation and will not set the X-Service-Catalog header.
insecure = False (BoolOpt) Verify HTTPS connections.
keyfile = None (StrOpt) Required if identity server requires client certificate
memcache_pool_conn_get_timeout = 10 (IntOpt) (Optional) Number of seconds that an operation will wait to get a memcached client connection from the pool.
memcache_pool_dead_retry = 300 (IntOpt) (Optional) Number of seconds memcached server is considered dead before it is tried again.
memcache_pool_maxsize = 10 (IntOpt) (Optional) Maximum total number of open connections to every memcached server.
memcache_pool_socket_timeout = 3 (IntOpt) (Optional) Socket timeout in seconds for communicating with a memcached server.
memcache_pool_unused_timeout = 60 (IntOpt) (Optional) Number of seconds a connection to memcached is held unused in the pool before it is closed.
memcache_secret_key = None (StrOpt) (Optional, mandatory if memcache_security_strategy is defined) This string is used for key derivation.
memcache_security_strategy = None (StrOpt) (Optional) If defined, indicate whether token data should be authenticated or authenticated and encrypted. Acceptable values are MAC or ENCRYPT. If MAC, token data is authenticated (with HMAC) in the cache. If ENCRYPT, token data is encrypted and authenticated in the cache. If the value is not one of these options or empty, auth_token will raise an exception on initialization.
memcache_use_advanced_pool = False (BoolOpt) (Optional) Use the advanced (eventlet safe) memcached client pool. The advanced pool will only work under python 2.x.
region_name = None (StrOpt) The region in which the identity server can be found.
revocation_cache_time = 10 (IntOpt) Determines the frequency at which the list of revoked tokens is retrieved from the Identity service (in seconds). A high number of revocation events combined with a low cache duration may significantly reduce performance.
signing_dir = None (StrOpt) Directory used to cache files related to PKI tokens.
token_cache_time = 300 (IntOpt) In order to prevent excessive effort spent validating tokens, the middleware caches previously-seen tokens for a configurable duration (in seconds). Set to -1 to disable caching completely.

Table 6.4. Description of clients configuration options

Configuration option = Default value Description
[cinder]
api_insecure = False (BoolOpt) Allow to perform insecure SSL requests to cinder.
api_version = 2 (IntOpt) Version of the Cinder API to use.
ca_file = None (StrOpt) Location of ca certificates file to use for cinder client requests.
endpoint_type = internalURL (StrOpt) Endpoint type for cinder client requests
[heat]
api_insecure = False (BoolOpt) Allow to perform insecure SSL requests to heat.
ca_file = None (StrOpt) Location of ca certificates file to use for heat client requests.
endpoint_type = internalURL (StrOpt) Endpoint type for heat client requests
[keystone]
api_insecure = False (BoolOpt) Allow to perform insecure SSL requests to keystone.
ca_file = None (StrOpt) Location of ca certificates file to use for keystone client requests.
endpoint_type = internalURL (StrOpt) Endpoint type for keystone client requests
[manila]
api_insecure = True (BoolOpt) Allow to perform insecure SSL requests to manila.
api_version = 1 (IntOpt) Version of the manila API to use.
ca_file = None (StrOpt) Location of ca certificates file to use for manila client requests.
[neutron]
api_insecure = False (BoolOpt) Allow to perform insecure SSL requests to neutron.
ca_file = None (StrOpt) Location of ca certificates file to use for neutron client requests.
endpoint_type = internalURL (StrOpt) Endpoint type for neutron client requests
[nova]
api_insecure = False (BoolOpt) Allow to perform insecure SSL requests to nova.
ca_file = None (StrOpt) Location of ca certificates file to use for nova client requests.
endpoint_type = internalURL (StrOpt) Endpoint type for nova client requests
[swift]
api_insecure = False (BoolOpt) Allow to perform insecure SSL requests to swift.
ca_file = None (StrOpt) Location of ca certificates file to use for swift client requests.
endpoint_type = internalURL (StrOpt) Endpoint type for swift client requests

Table 6.5. Description of common configuration options

Configuration option = Default value Description
[DEFAULT]
admin_project_domain_name = default (StrOpt) The name of the domain for the service project(ex. tenant).
admin_user_domain_name = default (StrOpt) The name of the domain to which the admin user belongs.
api_workers = 0 (IntOpt) Number of workers for Sahara API service (0 means all-in-one-thread configuration).
cleanup_time_for_incomplete_clusters = 0 (IntOpt) Maximal time (in hours) for clusters allowed to be in states other than "Active", "Deleting" or "Error". If a cluster is not in "Active", "Deleting" or "Error" state and last update of it was longer than "cleanup_time_for_incomplete_clusters" hours ago then it will be deleted automatically. (0 value means that automatic clean up is disabled).
cluster_operation_trust_expiration_hours = 24 (IntOpt) Defines the period of time (in hours) after which trusts created to allow sahara to create or scale a cluster will expire. Note that this value should be significantly larger than the value of the cleanup_time_for_incomplete_clusters configuration key if use of the cluster cleanup feature is desired (the trust must last at least as long as a cluster could validly take to stall in its creation, plus the timeout value set in that key, plus one hour for the period of the cleanup job).
cluster_remote_threshold = 70 (IntOpt) The same as global_remote_threshold, but for a single cluster.
compute_topology_file = etc/sahara/compute.topology (StrOpt) File with nova compute topology. It should contain mapping between nova computes and racks.
default_ntp_server = pool.ntp.org (StrOpt) Default ntp server for time sync
disable_event_log = False (BoolOpt) Disables event log feature.
enable_data_locality = False (BoolOpt) Enables data locality for hadoop cluster. Also enables data locality for Swift used by hadoop. If enabled, 'compute_topology' and 'swift_topology' configuration parameters should point to OpenStack and Swift topology correspondingly.
enable_hypervisor_awareness = True (BoolOpt) Enables four-level topology for data locality. Works only if corresponding plugin supports such mode.
enable_notifications = False (BoolOpt) Enables sending notifications to Ceilometer
executor_thread_pool_size = 64 (IntOpt) Size of executor thread pool.
global_remote_threshold = 100 (IntOpt) Maximum number of remote operations that will be running at the same time. Note that each remote operation requires its own process to run.
heat_stack_tags = data-processing-cluster (ListOpt) List of tags to be used during operating with stack.
infrastructure_engine = heat (StrOpt) An engine which will be used to provision infrastructure for Hadoop cluster.
job_binary_max_KB = 5120 (IntOpt) Maximum length of job binary data in kilobytes that may be stored or retrieved in a single operation.
job_canceling_timeout = 300 (IntOpt) Timeout for canceling job execution (in seconds). Sahara will try to cancel job execution during this time.
job_workflow_postfix = (StrOpt) Postfix for storing jobs in hdfs. Will be added to '/user/<hdfs user>/' path.
max_header_line = 16384 (IntOpt) Maximum line size of message headers to be accepted. max_header_line may need to be increased when using large tokens (typically those generated by the Keystone v3 API with big service catalogs).
memcached_servers = None (ListOpt) Memcached servers or None for in process cache.
min_transient_cluster_active_time = 30 (IntOpt) Minimal "lifetime" in seconds for a transient cluster. Cluster is guaranteed to be "alive" within this time period.
node_domain = novalocal (StrOpt) The suffix of the node's FQDN. In nova-network that is the dhcp_domain config parameter.
os_region_name = None (StrOpt) Region name used to get services endpoints.
periodic_enable = True (BoolOpt) Enable periodic tasks.
periodic_fuzzy_delay = 60 (IntOpt) Range in seconds to randomly delay when starting the periodic task scheduler to reduce stampeding. (Disable by setting to 0).
periodic_interval_max = 60 (IntOpt) Max interval size between periodic tasks execution in seconds.
plugins = vanilla, hdp, spark, cdh (ListOpt) List of plugins to be loaded. Sahara preserves the order of the list when returning it.
proxy_command = (StrOpt) Proxy command used to connect to instances. If set, this command should open a netcat socket, that Sahara will use for SSH and HTTP connections. Use {host} and {port} to describe the destination. Other available keywords: {tenant_id}, {network_id}, {router_id}.
remote = ssh (StrOpt) A method for Sahara to execute commands on VMs.
rootwrap_command = sudo sahara-rootwrap /etc/sahara/rootwrap.conf (StrOpt) Rootwrap command to leverage. Use in conjunction with use_rootwrap=True
swift_topology_file = etc/sahara/swift.topology (StrOpt) File with Swift topology.It should contain mapping between Swift nodes and racks.
use_external_key_manager = False (BoolOpt) Enable Sahara to use an external key manager service provided by the identity service catalog. Sahara will store all keys with the manager service.
use_floating_ips = True (BoolOpt) If set to True, Sahara will use floating IPs to communicate with instances. To make sure that all instances have floating IPs assigned in Nova Network set "auto_assign_floating_ip=True" in nova.conf. If Neutron is used for networking, make sure that all Node Groups have "floating_ip_pool" parameter defined.
use_identity_api_v3 = True (BoolOpt) Enables Sahara to use Keystone API v3. If that flag is disabled, per-job clusters will not be terminated automatically.
use_namespaces = False (BoolOpt) Use network namespaces for communication (only valid to use in conjunction with use_neutron=True).
use_neutron = False (BoolOpt) Use Neutron Networking (False indicates the use of Nova networking).
use_rootwrap = False (BoolOpt) Use rootwrap facility to allow non-root users to run the sahara-all server instance and access private network IPs (only valid to use in conjunction with use_namespaces=True)
[conductor]
use_local = True (BoolOpt) Perform sahara-conductor operations locally.
[keystone_authtoken]
memcached_servers = None (ListOpt) Optionally specify a list of memcached server(s) to use for caching. If left undefined, tokens will instead be cached in-process.

Table 6.6. Description of CORS configuration options

Configuration option = Default value Description
[cors]
allow_credentials = True (BoolOpt) Indicate that the actual request can include user credentials
allow_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which header field names may be used during the actual request.
allow_methods = GET, POST, PUT, DELETE, OPTIONS (ListOpt) Indicate which methods can be used during the actual request.
allowed_origin = None (StrOpt) Indicate whether this resource may be shared with the domain received in the requests "origin" header.
expose_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which headers are safe to expose to the API. Defaults to HTTP Simple Headers.
max_age = 3600 (IntOpt) Maximum cache age of CORS preflight requests.
[cors.subdomain]
allow_credentials = True (BoolOpt) Indicate that the actual request can include user credentials
allow_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which header field names may be used during the actual request.
allow_methods = GET, POST, PUT, DELETE, OPTIONS (ListOpt) Indicate which methods can be used during the actual request.
allowed_origin = None (StrOpt) Indicate whether this resource may be shared with the domain received in the requests "origin" header.
expose_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which headers are safe to expose to the API. Defaults to HTTP Simple Headers.
max_age = 3600 (IntOpt) Maximum cache age of CORS preflight requests.

Table 6.7. Description of database configuration options

Configuration option = Default value Description
[DEFAULT]
db_driver = sahara.db (StrOpt) Driver to use for database access.
[database]
backend = sqlalchemy (StrOpt) The back end to use for the database.
connection = None (StrOpt) The SQLAlchemy connection string to use to connect to the database.
connection_debug = 0 (IntOpt) Verbosity of SQL debugging information: 0=None, 100=Everything.
connection_trace = False (BoolOpt) Add Python stack traces to SQL as comment strings.
db_inc_retry_interval = True (BoolOpt) If True, increases the interval between retries of a database operation up to db_max_retry_interval.
db_max_retries = 20 (IntOpt) Maximum retries in case of connection error or deadlock error before error is raised. Set to -1 to specify an infinite retry count.
db_max_retry_interval = 10 (IntOpt) If db_inc_retry_interval is set, the maximum seconds between retries of a database operation.
db_retry_interval = 1 (IntOpt) Seconds between retries of a database transaction.
idle_timeout = 3600 (IntOpt) Timeout before idle SQL connections are reaped.
max_overflow = None (IntOpt) If set, use this value for max_overflow with SQLAlchemy.
max_pool_size = None (IntOpt) Maximum number of SQL connections to keep open in a pool.
max_retries = 10 (IntOpt) Maximum number of database connection retries during startup. Set to -1 to specify an infinite retry count.
min_pool_size = 1 (IntOpt) Minimum number of SQL connections to keep open in a pool.
mysql_sql_mode = TRADITIONAL (StrOpt) The SQL mode to be used for MySQL sessions. This option, including the default, overrides any server-set SQL mode. To use whatever SQL mode is set by the server configuration, set this to no value. Example: mysql_sql_mode=
pool_timeout = None (IntOpt) If set, use this value for pool_timeout with SQLAlchemy.
retry_interval = 10 (IntOpt) Interval between retries of opening a SQL connection.
slave_connection = None (StrOpt) The SQLAlchemy connection string to use to connect to the slave database.
sqlite_db = oslo.sqlite (StrOpt) The file name to use with SQLite.
sqlite_synchronous = True (BoolOpt) If True, SQLite uses synchronous mode.
use_db_reconnect = False (BoolOpt) Enable the experimental use of database reconnect on connection lost.

Table 6.8. Description of domain configuration options

Configuration option = Default value Description
[DEFAULT]
proxy_user_domain_name = None (StrOpt) The domain Sahara will use to create new proxy users for Swift object access.
proxy_user_role_names = Member (ListOpt) A list of the role names that the proxy user should assume through trust for Swift object access.
use_domain_for_proxy_users = False (BoolOpt) Enables Sahara to use a domain for creating temporary proxy users to access Swift. If this is enabled a domain must be created for Sahara to use.

Table 6.9. Description of logging configuration options

Configuration option = Default value Description
[DEFAULT]
debug = False (BoolOpt) Print debugging output (set logging level to DEBUG instead of default INFO level).
default_log_levels = amqplib=WARN, qpid.messaging=INFO, stevedore=INFO, eventlet.wsgi.server=WARN, sqlalchemy=WARN, boto=WARN, suds=INFO, keystone=INFO, paramiko=WARN, requests=WARN, iso8601=WARN, oslo_messaging=INFO, neutronclient=INFO (ListOpt) List of logger=LEVEL pairs. This option is ignored if log_config_append is set.
fatal_deprecations = False (BoolOpt) Enables or disables fatal status of deprecations.
instance_format = "[instance: %(uuid)s] " (StrOpt) The format for an instance that is passed with the log message.
instance_uuid_format = "[instance: %(uuid)s] " (StrOpt) The format for an instance UUID that is passed with the log message.
log_config_append = None (StrOpt) The name of a logging configuration file. This file is appended to any existing logging configuration files. For details about logging configuration files, see the Python logging module documentation. Note that when logging configuration files are used then all logging configuration is set in the configuration file and other logging configuration options are ignored (for example, log_format).
log_date_format = %Y-%m-%d %H:%M:%S (StrOpt) Format string for %%(asctime)s in log records. Default: %(default)s . This option is ignored if log_config_append is set.
log_dir = None (StrOpt) (Optional) The base directory used for relative --log-file paths. This option is ignored if log_config_append is set.
log_file = None (StrOpt) (Optional) Name of log file to output to. If no default is set, logging will go to stdout. This option is ignored if log_config_append is set.
log_format = None (StrOpt) DEPRECATED. A logging.Formatter log message format string which may use any of the available logging.LogRecord attributes. This option is deprecate, use logging_context_format_string and logging_default_format_string instead. This option is ignored if log_config_append is set.
logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s (StrOpt) Format string to use for log messages with context.
logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d (StrOpt) Data to append to log format when level is DEBUG.
logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s (StrOpt) Format string to use for log messages without context.
logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s (StrOpt) Prefix each line of exception output with this format.
publish_errors = False (BoolOpt) Enables or disables publication of error events.
syslog_log_facility = LOG_USER (StrOpt) Syslog facility to receive log lines. This option is ignored if log_config_append is set.
use_stderr = True (BoolOpt) Log output to standard error. This option is ignored if log_config_append is set.
use_syslog = False (BoolOpt) Use syslog for logging. Existing syslog format is DEPRECATED and will be changed later to honor RFC5424. This option is ignored if log_config_append is set.
use_syslog_rfc_format = True (BoolOpt) (Optional) Enables or disables syslog rfc5424 format for logging. If enabled, prefixes the MSG part of the syslog message with APP-NAME (RFC5424). The format without the APP-NAME is deprecated in Kilo, and will be removed in Mitaka, along with this option. This option is ignored if log_config_append is set.
verbose = True (BoolOpt) If set to false, will disable INFO logging level, making WARNING the default.
watch_log_file = False (BoolOpt) (Optional) Uses logging handler designed to watch file system. When log file is moved or removed this handler will open a new log file with specified path instantaneously. It makes sense only if log-file option is specified and Linux platform is used. This option is ignored if log_config_append is set.

Table 6.10. Description of Auth options for Swift access for VM configuration options

Configuration option = Default value Description
[object_store_access]
public_identity_ca_file = None (StrOpt) Location of ca certificate file to use for identity client requests via public endpoint
public_object_store_ca_file = None (StrOpt) Location of ca certificate file to use for object-store client requests via public endpoint

Table 6.11. Description of policy configuration options

Configuration option = Default value Description
[oslo_policy]
policy_default_rule = default (StrOpt) Default rule. Enforced when a requested rule is not found.
policy_dirs = ['policy.d'] (MultiStrOpt) Directories where policy configuration files are stored. They can be relative to any directory in the search path defined by the config_dir option, or absolute paths. The file defined by policy_file must exist for these directories to be searched. Missing or empty directories are ignored.
policy_file = policy.json (StrOpt) The JSON file that defines policies.

Table 6.12. Description of Qpid configuration options

Configuration option = Default value Description
[oslo_messaging_qpid]
amqp_auto_delete = False (BoolOpt) Auto-delete queues in AMQP.
amqp_durable_queues = False (BoolOpt) Use durable queues in AMQP.
qpid_heartbeat = 60 (IntOpt) Seconds between connection keepalive heartbeats.
qpid_hostname = localhost (StrOpt) Qpid broker hostname.
qpid_hosts = $qpid_hostname:$qpid_port (ListOpt) Qpid HA cluster host:port pairs.
qpid_password = (StrOpt) Password for Qpid connection.
qpid_port = 5672 (IntOpt) Qpid broker port.
qpid_protocol = tcp (StrOpt) Transport to use, either 'tcp' or 'ssl'.
qpid_receiver_capacity = 1 (IntOpt) The number of prefetched messages held by receiver.
qpid_sasl_mechanisms = (StrOpt) Space separated list of SASL mechanisms to use for auth.
qpid_tcp_nodelay = True (BoolOpt) Whether to disable the Nagle algorithm.
qpid_topology_version = 1 (IntOpt) The qpid topology version to use. Version 1 is what was originally used by impl_qpid. Version 2 includes some backwards-incompatible changes that allow broker federation to work. Users should update to version 2 when they are able to take everything down, as it requires a clean break.
qpid_username = (StrOpt) Username for Qpid connection.
send_single_reply = False (BoolOpt) Send a single AMQP reply to call message. The current behavior since oslo-incubator is to send two AMQP replies - first one with the payload, a second one to ensure the other has finished to send the payload. We are going to remove it in the N release, but we must keep backward compatible at the same time. This option provides such compatibility - it defaults to False in Liberty and can be turned on for early adopters with new installations or for testing. This option will be removed in the Mitaka release.

Table 6.13. Description of RabbitMQ configuration options

Configuration option = Default value Description
[oslo_messaging_rabbit]
amqp_auto_delete = False (BoolOpt) Auto-delete queues in AMQP.
amqp_durable_queues = False (BoolOpt) Use durable queues in AMQP.
fake_rabbit = False (BoolOpt) Deprecated, use rpc_backend=kombu+memory or rpc_backend=fake
heartbeat_rate = 2 (IntOpt) How often times during the heartbeat_timeout_threshold we check the heartbeat.
heartbeat_timeout_threshold = 60 (IntOpt) Number of seconds after which the Rabbit broker is considered down if heartbeat's keep-alive fails (0 disable the heartbeat). EXPERIMENTAL
kombu_reconnect_delay = 1.0 (FloatOpt) How long to wait before reconnecting in response to an AMQP consumer cancel notification.
kombu_reconnect_timeout = 60 (IntOpt) How long to wait before considering a reconnect attempt to have failed. This value should not be longer than rpc_response_timeout.
kombu_ssl_ca_certs = (StrOpt) SSL certification authority file (valid only if SSL enabled).
kombu_ssl_certfile = (StrOpt) SSL cert file (valid only if SSL enabled).
kombu_ssl_keyfile = (StrOpt) SSL key file (valid only if SSL enabled).
kombu_ssl_version = (StrOpt) SSL version to use (valid only if SSL enabled). Valid values are TLSv1 and SSLv23. SSLv2, SSLv3, TLSv1_1, and TLSv1_2 may be available on some distributions.
rabbit_ha_queues = False (BoolOpt) Use HA queues in RabbitMQ (x-ha-policy: all). If you change this option, you must wipe the RabbitMQ database.
rabbit_host = localhost (StrOpt) The RabbitMQ broker address where a single node is used.
rabbit_hosts = $rabbit_host:$rabbit_port (ListOpt) RabbitMQ HA cluster host:port pairs.
rabbit_login_method = AMQPLAIN (StrOpt) The RabbitMQ login method.
rabbit_max_retries = 0 (IntOpt) Maximum number of RabbitMQ connection retries. Default is 0 (infinite retry count).
rabbit_password = guest (StrOpt) The RabbitMQ password.
rabbit_port = 5672 (IntOpt) The RabbitMQ broker port where a single node is used.
rabbit_retry_backoff = 2 (IntOpt) How long to backoff for between retries when connecting to RabbitMQ.
rabbit_retry_interval = 1 (IntOpt) How frequently to retry connecting with RabbitMQ.
rabbit_use_ssl = False (BoolOpt) Connect over SSL for RabbitMQ.
rabbit_userid = guest (StrOpt) The RabbitMQ userid.
rabbit_virtual_host = / (StrOpt) The RabbitMQ virtual host.
send_single_reply = False (BoolOpt) Send a single AMQP reply to call message. The current behavior since oslo-incubator is to send two AMQP replies - first one with the payload, a second one to ensure the other has finished to send the payload. We are going to remove it in the N release, but we must keep backward compatible at the same time. This option provides such compatibility - it defaults to False in Liberty and can be turned on for early adopters with new installations or for testing. This option will be removed in the Mitaka release.

Table 6.14. Description of Redis configuration options

Configuration option = Default value Description
[DEFAULT]
host = 127.0.0.1 (StrOpt) Host to locate redis.
password = (StrOpt) Password for Redis server (optional).
port = 6379 (IntOpt) Use this port to connect to redis host.
[matchmaker_redis]
host = 127.0.0.1 (StrOpt) Host to locate redis.
password = (StrOpt) Password for Redis server (optional).
port = 6379 (IntOpt) Use this port to connect to redis host.

Table 6.15. Description of RPC configuration options

Configuration option = Default value Description
[DEFAULT]
rpc_backend = rabbit (StrOpt) The messaging driver to use, defaults to rabbit. Other drivers include qpid and zmq.
rpc_cast_timeout = 30 (IntOpt) Seconds to wait before a cast expires (TTL). Only supported by impl_zmq.
rpc_conn_pool_size = 30 (IntOpt) Size of RPC connection pool.
rpc_poll_timeout = 1 (IntOpt) The default number of seconds that poll should wait. Poll raises timeout exception when timeout expired.
rpc_response_timeout = 60 (IntOpt) Seconds to wait for a response from a call.
[oslo_concurrency]
disable_process_locking = False (BoolOpt) Enables or disables inter-process locks.
lock_path = None (StrOpt) Directory to use for lock files. For security, the specified directory should only be writable by the user running the processes that need locking. Defaults to environment variable OSLO_LOCK_PATH. If external locks are used, a lock path must be set.
[oslo_messaging_amqp]
allow_insecure_clients = False (BoolOpt) Accept clients using either SSL or plain TCP
broadcast_prefix = broadcast (StrOpt) address prefix used when broadcasting to all servers
container_name = None (StrOpt) Name for the AMQP container
group_request_prefix = unicast (StrOpt) address prefix when sending to any server in group
idle_timeout = 0 (IntOpt) Timeout for inactive connections (in seconds)
password = (StrOpt) Password for message broker authentication
sasl_config_dir = (StrOpt) Path to directory that contains the SASL configuration
sasl_config_name = (StrOpt) Name of configuration file (without .conf suffix)
sasl_mechanisms = (StrOpt) Space separated list of acceptable SASL mechanisms
server_request_prefix = exclusive (StrOpt) address prefix used when sending to a specific server
ssl_ca_file = (StrOpt) CA certificate PEM file to verify server certificate
ssl_cert_file = (StrOpt) Identifying certificate PEM file to present to clients
ssl_key_file = (StrOpt) Private key PEM file used to sign cert_file certificate
ssl_key_password = None (StrOpt) Password for decrypting ssl_key_file (if encrypted)
trace = False (BoolOpt) Debug: dump AMQP frames to stdout
username = (StrOpt) User name for message broker authentication

Table 6.16. Description of timeouts configuration options

Configuration option = Default value Description
[timeouts]
await_attach_volumes = 10 (IntOpt) Wait for attaching volumes to instances, in seconds
await_for_instances_active = 10800 (IntOpt) Wait for instances to become active, in seconds
delete_instances_timeout = 10800 (IntOpt) Wait for instances to be deleted, in seconds
detach_volume_timeout = 300 (IntOpt) Timeout for detaching volumes from instance, in seconds
ips_assign_timeout = 10800 (IntOpt) Assign IPs timeout, in seconds
volume_available_timeout = 10800 (IntOpt) Wait for volumes to become available, in seconds
wait_until_accessible = 10800 (IntOpt) Wait for instance accessibility, in seconds

6.1. New, updated, and deprecated options in Liberty for Data Processing service

Table 6.17. New options

Option = default value (Type) Help string
[DEFAULT] cluster_operation_trust_expiration_hours = 24 (IntOpt) Defines the period of time (in hours) after which trusts created to allow sahara to create or scale a cluster will expire. Note that this value should be significantly larger than the value of the cleanup_time_for_incomplete_clusters configuration key if use of the cluster cleanup feature is desired (the trust must last at least as long as a cluster could validly take to stall in its creation, plus the timeout value set in that key, plus one hour for the period of the cleanup job).
[DEFAULT] default_ntp_server = pool.ntp.org (StrOpt) Default ntp server for time sync
[DEFAULT] executor_thread_pool_size = 64 (IntOpt) Size of executor thread pool.
[DEFAULT] heat_stack_tags = data-processing-cluster (ListOpt) List of tags to be used during operating with stack.
[DEFAULT] host = 127.0.0.1 (StrOpt) Host to locate redis.
[DEFAULT] password = (StrOpt) Password for Redis server (optional).
[DEFAULT] port = 6379 (IntOpt) Use this port to connect to redis host.
[DEFAULT] rpc_conn_pool_size = 30 (IntOpt) Size of RPC connection pool.
[DEFAULT] rpc_poll_timeout = 1 (IntOpt) The default number of seconds that poll should wait. Poll raises timeout exception when timeout expired.
[DEFAULT] rpc_zmq_all_req_rep = True (BoolOpt) Use REQ/REP pattern for all methods CALL/CAST/FANOUT.
[DEFAULT] rpc_zmq_concurrency = eventlet (StrOpt) Type of concurrency used. Either "native" or "eventlet"
[DEFAULT] watch_log_file = False (BoolOpt) (Optional) Uses logging handler designed to watch file system. When log file is moved or removed this handler will open a new log file with specified path instantaneously. It makes sense only if log-file option is specified and Linux platform is used. This option is ignored if log_config_append is set.
[DEFAULT] zmq_use_broker = True (BoolOpt) Shows whether zmq-messaging uses broker or not.
[cinder] endpoint_type = internalURL (StrOpt) Endpoint type for cinder client requests
[cors] allow_credentials = True (BoolOpt) Indicate that the actual request can include user credentials
[cors] allow_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which header field names may be used during the actual request.
[cors] allow_methods = GET, POST, PUT, DELETE, OPTIONS (ListOpt) Indicate which methods can be used during the actual request.
[cors] allowed_origin = None (StrOpt) Indicate whether this resource may be shared with the domain received in the requests "origin" header.
[cors] expose_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which headers are safe to expose to the API. Defaults to HTTP Simple Headers.
[cors] max_age = 3600 (IntOpt) Maximum cache age of CORS preflight requests.
[cors.subdomain] allow_credentials = True (BoolOpt) Indicate that the actual request can include user credentials
[cors.subdomain] allow_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which header field names may be used during the actual request.
[cors.subdomain] allow_methods = GET, POST, PUT, DELETE, OPTIONS (ListOpt) Indicate which methods can be used during the actual request.
[cors.subdomain] allowed_origin = None (StrOpt) Indicate whether this resource may be shared with the domain received in the requests "origin" header.
[cors.subdomain] expose_headers = Content-Type, Cache-Control, Content-Language, Expires, Last-Modified, Pragma (ListOpt) Indicate which headers are safe to expose to the API. Defaults to HTTP Simple Headers.
[cors.subdomain] max_age = 3600 (IntOpt) Maximum cache age of CORS preflight requests.
[heat] endpoint_type = internalURL (StrOpt) Endpoint type for heat client requests
[keystone] endpoint_type = internalURL (StrOpt) Endpoint type for keystone client requests
[keystone_authtoken] region_name = None (StrOpt) The region in which the identity server can be found.
[manila] api_insecure = True (BoolOpt) Allow to perform insecure SSL requests to manila.
[manila] api_version = 1 (IntOpt) Version of the manila API to use.
[manila] ca_file = None (StrOpt) Location of ca certificates file to use for manila client requests.
[neutron] endpoint_type = internalURL (StrOpt) Endpoint type for neutron client requests
[nova] endpoint_type = internalURL (StrOpt) Endpoint type for nova client requests
[object_store_access] public_identity_ca_file = None (StrOpt) Location of ca certificate file to use for identity client requests via public endpoint
[object_store_access] public_object_store_ca_file = None (StrOpt) Location of ca certificate file to use for object-store client requests via public endpoint
[oslo_messaging_amqp] password = (StrOpt) Password for message broker authentication
[oslo_messaging_amqp] sasl_config_dir = (StrOpt) Path to directory that contains the SASL configuration
[oslo_messaging_amqp] sasl_config_name = (StrOpt) Name of configuration file (without .conf suffix)
[oslo_messaging_amqp] sasl_mechanisms = (StrOpt) Space separated list of acceptable SASL mechanisms
[oslo_messaging_amqp] username = (StrOpt) User name for message broker authentication
[oslo_messaging_qpid] send_single_reply = False (BoolOpt) Send a single AMQP reply to call message. The current behavior since oslo-incubator is to send two AMQP replies - first one with the payload, a second one to ensure the other has finished to send the payload. We are going to remove it in the N release, but we must keep backward compatible at the same time. This option provides such compatibility - it defaults to False in Liberty and can be turned on for early adopters with new installations or for testing. This option will be removed in the Mitaka release.
[oslo_messaging_rabbit] kombu_reconnect_timeout = 60 (IntOpt) How long to wait before considering a reconnect attempt to have failed. This value should not be longer than rpc_response_timeout.
[oslo_messaging_rabbit] send_single_reply = False (BoolOpt) Send a single AMQP reply to call message. The current behavior since oslo-incubator is to send two AMQP replies - first one with the payload, a second one to ensure the other has finished to send the payload. We are going to remove it in the N release, but we must keep backward compatible at the same time. This option provides such compatibility - it defaults to False in Liberty and can be turned on for early adopters with new installations or for testing. This option will be removed in the Mitaka release.
[oslo_middleware] secure_proxy_ssl_header = X-Forwarded-Proto (StrOpt) The HTTP Header that will be used to determine what the original request protocol scheme was, even if it was hidden by an SSL termination proxy.
[retries] retries_number = 5 (IntOpt) Number of times to retry the request to client before failing
[retries] retry_after = 10 (IntOpt) Time between the retries to client (in seconds).
[swift] endpoint_type = internalURL (StrOpt) Endpoint type for swift client requests

Table 6.18. New default values

Option Previous default value New default value
[DEFAULT] default_log_levels amqplib=WARN, qpid.messaging=INFO, stevedore=INFO, eventlet.wsgi.server=WARN, sqlalchemy=WARN, boto=WARN, suds=INFO, keystone=INFO, paramiko=WARN, requests=WARN, iso8601=WARN, oslo_messaging=INFO amqplib=WARN, qpid.messaging=INFO, stevedore=INFO, eventlet.wsgi.server=WARN, sqlalchemy=WARN, boto=WARN, suds=INFO, keystone=INFO, paramiko=WARN, requests=WARN, iso8601=WARN, oslo_messaging=INFO, neutronclient=INFO
[DEFAULT] infrastructure_engine direct heat
[DEFAULT] logging_exception_prefix %(asctime)s.%(msecs)03d %(process)d TRACE %(name)s %(instance)s %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s
[DEFAULT] rpc_zmq_matchmaker local redis
[DEFAULT] use_syslog_rfc_format False True
[DEFAULT] verbose False True
[matchmaker_redis] password None
[oslo_messaging_rabbit] heartbeat_timeout_threshold 0 60

Table 6.19. Deprecated options

Deprecated option New Option
[DEFAULT] rpc_thread_pool_size [DEFAULT] executor_thread_pool_size
[DEFAULT] log_format None
[DEFAULT] use_syslog None