Chapter 12. Database Security
The choice of database server is an important consideration in the security of an OpenStack deployment. Multiple factors should be considered when deciding on a database server, however for the scope of this book only security considerations will be discussed. OpenStack supports a variety of database types, with this chapter focusing on MySQL.
12.1. Database access control
Each of the core OpenStack services (such as nova, keystone, neutron, cinder) store their state and configuration information in databases. This chapter examines how databases are used currently in OpenStack, and also explores security concerns, and the security ramifications of certain database back end choices.
12.2. OpenStack’s database access model
All of the services within an OpenStack project access a single logical database, per OpenStack service. There are currently no reference policies for creating table or row based access restrictions to the database.
There are no general provisions for granular control of database operations in OpenStack. Access and privileges are granted simply based on whether a node has access to the database or not. In this scenario, nodes with access to the database might have full privileges to
12.2.1. Granular access control
By default, each of the OpenStack services and their processes access the database using a shared set of credentials. This makes auditing database operations and revoking access privileges from a service and its processes to the database particularly difficult.
12.2.2. Proxy database connections
The Compute nodes are the least trusted of the services because they host project instances. The
nova-conductor service serves as a database proxy, acting as an intermediary between the Compute nodes and the database. The ramifications are examined later in this chapter.
Consider the following practices:
- Isolate all database communications to a management network.
- Encrypt communications using TLS.
- Create unique database user accounts for each OpenStack service endpoint (illustrated below). For Red Hat OpenStack Platform, the director randomly generates a unique password for each service during deployment.
12.3. Database authentication and access control
Given the risks around access to the database, consider creating unique database user accounts for each node needing access to the database. This practice enables better analysis and auditing for determining compliance, or allows you to isolate hosts in the event of an incident. When creating these per service endpoint database user accounts, care should be taken to ensure that they are configured to require TLS. Alternatively, for increased security it is recommended that the database accounts be configured using X.509 certificate authentication in addition to user names and passwords.
12.4. Limiting privileges
A separate database administrator (DBA) account should be created and protected, as it will have full privileges to create/drop databases, create user accounts, and update user privileges. This allows you to separate the scope of responsibility, and consequently helps mitigate the risk of accidental misconfiguration or deliberate incidents.
The database user accounts for the OpenStack services and for each node should have privileges limited to just the database relevant to the service where the node is a member.
12.4.1. Require SSL transport for user accounts
Configuration example for MySQL:
GRANT ALL ON dbname.* to 'compute01'@'hostname' IDENTIFIED BY 'NOVA_DBPASS' REQUIRE SSL;
12.4.2. Service configuration
If your database server is configured for TLS transport, you will need to specify the certificate authority information for use with the initial connection string in the SQLAlchemy query.
This example demonstrates a
sql_connection string to MySQL. Note that the exact file path might vary for containerized services.
sql_connection = mysql://compute01:NOVA_DBPASS@localhost/nova?read_default_group=tripleo&read_default_file=/etc/my.cnf.d/tripleo.cnf
This setting forces SQLAlchemy’s MySQL backend (PyMySQL) to read an extra file on disk which contains any specific connection configuration. For example:
cat /etc/my.cnf.d/tripleo.cnf [tripleo] bind-address=22.214.171.124 ssl=1 ssl-ca=/etc/ipa/ca.crt
This is used to hide the bind-address which is used by MySQL clients to connect to the database from that host. This is also used to enable SSL connection and specify certs, the CA, among others.
As a result of the
tripleo.cnf configuration, nova never stores connection-specific details into its own internal nova database. So enabling or changing the SSL settings should not impact nova services.
12.5. Authentication with X.509 certificates
Security could be enhanced by requiring X.509 client certificates for authentication. This approach provides additional assurance of the identity of the connecting client, and checks that the communications are encrypted.
Configuration example for MySQL:
GRANT ALL on dbname.* to 'compute01'@'hostname' IDENTIFIED BY 'NOVA_DBPASS' REQUIRE SUBJECT '/C=XX/ST=YYY/L=ZZZZ/O=cloudycloud/CN=compute01' AND ISSUER '/C=XX/ST=YYY/L=ZZZZ/O=cloudycloud/CN=cloud-ca';
12.6. Service database configuration
If your database server is configured to require X.509 certificates for authentication, you will need to specify the appropriate SQLAlchemy query parameters for the database back end. These parameters specify the certificate, private key, and certificate authority information for use with the initial connection string.
The SSL configuration should be populated in
tripleo.cnf for each service. When
InternalTLS is enabled, this is automatically done by director.
This example demonstrates a
sql_connection string for X.509 certificate authentication to MySQL:
sql_connection = mysql://compute01:NOVA_DBPASS@localhost/nova? charset=utf8&ssl_ca = /etc/mysql/cacert.pem&ssl_cert=/etc/mysql/server-cert.pem&ssl_key=/etc/mysql/server-key.pem
12.7. Using nova-conductor to proxy connections
Compute database connections are proxied using a sub-service called
nova-conductor. The primary purpose of this approach is to meet data persistence needs, as opposed to directly communicating with the database.
Nova-conductor receives requests over RPC and performs actions on behalf of the calling service without granting granular access to the database, its tables, or the data residing within. Nova-conductor essentially abstracts direct database access away from compute nodes.
This abstraction offers the advantage of restricting services to executing methods with parameters (similar to stored procedures), and helps avoid having a large number of systems directly accessing or modifying database data. This approach avoids having the procedures stored or executed within the context or scope of the database itself, a frequent criticism of typical stored procedures.
This approach limits the possibility of more fine-grained access control, and complicates data access auditing. Because the nova-conductor service receives requests over RPC, it highlights the importance of improving the security of messaging. Any node with access to the message queue could execute these methods provided by the
nova-conductor and effectively modify the database.
nova-conductor only applies to Compute, direct database access from Compute nodes might still be necessary for the operation of other OpenStack components such as Telemetry (ceilometer), neutron, and cinder.
12.8. Database transport security
This chapter covers issues related to network communications to and from the database server. This includes IP address bindings and encrypting network traffic with TLS.
12.8.1. Database server IP address binding
To help isolate sensitive database communications between the services and the database, consider configuring the database server(s) to only allow communications to and from the database over an isolated management network. This is done by restricting the interface or IP address on which the database server binds a network socket for incoming client connections:
The director configures MySQL to restrict the bind IP address. You can view the configuration in
/var/lib/config-data/puppet-generated/mysql/etc/my.cnf.d/galera.cnf. The service is exposed through the
internalapi setting, for example:
internalapi: bind-address = overcloud-controller-0.internalapi.localdomain.
For HA deployments, the galera-specific replication traffic is configured to use the same
internalapi network from
galera.cnf. For example:
wsrep_cluster_address = gcomm://controller-0.internalapi.lab.local,controller-1.internalapi.lab.local,controller-2.internalapi.lab.local
bind-address setting is different from what it configured in each client’s
tripleo.cnf configures sqlalchemy to use a specific IP as the source address when creating a socket to connect to the database. This is required to ensure the connection is not impacted by VIP failover. For more information, see https://bugzilla.redhat.com/show_bug.cgi?id=1378008.
12.9. Encrypting database communications
In addition to restricting database communications to the management network, consider configuring your database back end to require TLS for client connections. This approach helps protect the communications from tampering and eavesdropping. TLS also provides the framework for performing database user authentication through X.509 certificates (commonly referred to as PKI). See below for guidance on how TLS is typically configured for MySQL.
For Red Hat OpenStack Platform deployments, the director automatically applies security hardening settings. This includes file permissions for certificates and key files, with ownership restricted to the database daemon user. This helps prevent unauthorized access from other processes and users on the database server. For more information, see https://blueprints.launchpad.net/tripleo/+spec/tls-via-certmonger
You can optionally restrict the set of SSL ciphers used for the encrypted connection. See
ciphers for a list of ciphers and the syntax for specifying the cipher string:
ssl-cipher = 'cipher:list'
When TLSEverywhere is enabled, Red Hat OpenStack Platform 13 is able to pass the cipher list in mysqld’s configuration file:
/var/lib/config-data/puppet-generated/mysql/etc/my.cnf.d/galera.cnf. By default, TLSv1.0 ciphers are disabled. If required, you can override the default with the
gcomm_cipher Heat parameters. For more information, see https://review.openstack.org/#/c/566509.