Chapter 12. Database Security

The choice of database server is an important consideration in the security of an OpenStack deployment. Multiple factors should be considered when deciding on a database server, however for the scope of this book only security considerations will be discussed. OpenStack supports a variety of database types, with this chapter focusing on MySQL.

12.1. Database access control

Each of the core OpenStack services (such as nova, keystone, neutron, cinder) store their state and configuration information in databases. This chapter examines how databases are used currently in OpenStack, and also explores security concerns, and the security ramifications of certain database back end choices.

12.2. OpenStack’s database access model

All of the services within an OpenStack project access a single logical database, per OpenStack service. There are currently no reference policies for creating table or row based access restrictions to the database.

There are no general provisions for granular control of database operations in OpenStack. Access and privileges are granted simply based on whether a node has access to the database or not. In this scenario, nodes with access to the database might have full privileges to DROP, INSERT, or UPDATE functions.

12.2.1. Granular access control

By default, each of the OpenStack services and their processes access the database using a shared set of credentials. This makes auditing database operations and revoking access privileges from a service and its processes to the database particularly difficult.

databaseusername

12.2.2. Proxy database connections

The Compute nodes are the least trusted of the services because they host project instances. The nova-conductor service serves as a database proxy, acting as an intermediary between the Compute nodes and the database. The ramifications are examined later in this chapter.

Consider the following practices:

  • Isolate all database communications to a management network.
  • Encrypt communications using TLS.
  • Create unique database user accounts for each OpenStack service endpoint (illustrated below). For Red Hat OpenStack Platform, the director randomly generates a unique password for each service during deployment.

databaseusernamessl

12.3. Database authentication and access control

Given the risks around access to the database, consider creating unique database user accounts for each node needing access to the database. This practice enables better analysis and auditing for determining compliance, or allows you to isolate hosts in the event of an incident. When creating these per service endpoint database user accounts, care should be taken to ensure that they are configured to require TLS. Alternatively, for increased security it is recommended that the database accounts be configured using X.509 certificate authentication in addition to user names and passwords.

12.4. Limiting privileges

A separate database administrator (DBA) account should be created and protected, as it will have full privileges to create/drop databases, create user accounts, and update user privileges. This allows you to separate the scope of responsibility, and consequently helps mitigate the risk of accidental misconfiguration or deliberate incidents.

The database user accounts for the OpenStack services and for each node should have privileges limited to just the database relevant to the service where the node is a member.

12.4.1. Require SSL transport for user accounts

  • Configuration example for MySQL:

    GRANT ALL ON dbname.* to 'compute01'@'hostname' IDENTIFIED BY 'NOVA_DBPASS' REQUIRE SSL;

12.4.2. Service configuration

If your database server is configured for TLS transport, you will need to specify the certificate authority information for use with the initial connection string in the SQLAlchemy query.

This example demonstrates a sql_connection string to MySQL. Note that the exact file path might vary for containerized services.

sql_connection = mysql://compute01:NOVA_DBPASS@localhost/nova?read_default_group=tripleo&read_default_file=/etc/my.cnf.d/tripleo.cnf

This setting forces SQLAlchemy’s MySQL backend (PyMySQL) to read an extra file on disk which contains any specific connection configuration. For example:

cat /etc/my.cnf.d/tripleo.cnf
[tripleo]
bind-address=172.10.1.10
ssl=1
ssl-ca=/etc/ipa/ca.crt

This is used to hide the bind-address which is used by MySQL clients to connect to the database from that host. This is also used to enable SSL connection and specify certs, the CA, among others.

As a result of the tripleo.cnf configuration, nova never stores connection-specific details into its own internal nova database. So enabling or changing the SSL settings should not impact nova services.

12.5. Authentication with X.509 certificates

Security could be enhanced by requiring X.509 client certificates for authentication. This approach provides additional assurance of the identity of the connecting client, and checks that the communications are encrypted.

  • Configuration example for MySQL:

    GRANT ALL on dbname.* to 'compute01'@'hostname' IDENTIFIED BY 'NOVA_DBPASS' REQUIRE SUBJECT
    '/C=XX/ST=YYY/L=ZZZZ/O=cloudycloud/CN=compute01' AND ISSUER
    '/C=XX/ST=YYY/L=ZZZZ/O=cloudycloud/CN=cloud-ca';

12.6. Service database configuration

If your database server is configured to require X.509 certificates for authentication, you will need to specify the appropriate SQLAlchemy query parameters for the database back end. These parameters specify the certificate, private key, and certificate authority information for use with the initial connection string.

The SSL configuration should be populated in tripleo.cnf for each service. When InternalTLS is enabled, this is automatically done by director.

This example demonstrates a sql_connection string for X.509 certificate authentication to MySQL:

sql_connection = mysql://compute01:NOVA_DBPASS@localhost/nova?
charset=utf8&ssl_ca = /etc/mysql/cacert.pem&ssl_cert=/etc/mysql/server-cert.pem&ssl_key=/etc/mysql/server-key.pem

12.7. Using nova-conductor to proxy connections

Compute database connections are proxied using a sub-service called nova-conductor. The primary purpose of this approach is to meet data persistence needs, as opposed to directly communicating with the database.

Nova-conductor receives requests over RPC and performs actions on behalf of the calling service without granting granular access to the database, its tables, or the data residing within. Nova-conductor essentially abstracts direct database access away from compute nodes.

This abstraction offers the advantage of restricting services to executing methods with parameters (similar to stored procedures), and helps avoid having a large number of systems directly accessing or modifying database data. This approach avoids having the procedures stored or executed within the context or scope of the database itself, a frequent criticism of typical stored procedures.

novaconductor

Note

This approach limits the possibility of more fine-grained access control, and complicates data access auditing. Because the nova-conductor service receives requests over RPC, it highlights the importance of improving the security of messaging. Any node with access to the message queue could execute these methods provided by the nova-conductor and effectively modify the database.

Since nova-conductor only applies to Compute, direct database access from Compute nodes might still be necessary for the operation of other OpenStack components such as Telemetry (ceilometer), neutron, and cinder.

12.8. Database transport security

This chapter covers issues related to network communications to and from the database server. This includes IP address bindings and encrypting network traffic with TLS.

12.8.1. Database server IP address binding

To help isolate sensitive database communications between the services and the database, consider configuring the database server(s) to only allow communications to and from the database over an isolated management network. This is done by restricting the interface or IP address on which the database server binds a network socket for incoming client connections:

The director configures MySQL to restrict the bind IP address. You can view the configuration in /var/lib/config-data/puppet-generated/mysql/etc/my.cnf.d/galera.cnf. The service is exposed through the internalapi setting, for example: internalapi: bind-address = overcloud-controller-0.internalapi.localdomain.

For HA deployments, the galera-specific replication traffic is configured to use the same internalapi network from galera.cnf. For example:

wsrep_cluster_address = gcomm://controller-0.internalapi.lab.local,controller-1.internalapi.lab.local,controller-2.internalapi.lab.local
Note

This bind-address setting is different from what it configured in each client’s tripleo.cnf. The tripleo.cnf configures sqlalchemy to use a specific IP as the source address when creating a socket to connect to the database. This is required to ensure the connection is not impacted by VIP failover. For more information, see https://bugzilla.redhat.com/show_bug.cgi?id=1378008.

12.9. Encrypting database communications

In addition to restricting database communications to the management network, consider configuring your database back end to require TLS for client connections. This approach helps protect the communications from tampering and eavesdropping. TLS also provides the framework for performing database user authentication through X.509 certificates (commonly referred to as PKI). See below for guidance on how TLS is typically configured for MySQL.

For Red Hat OpenStack Platform deployments, the director automatically applies security hardening settings. This includes file permissions for certificates and key files, with ownership restricted to the database daemon user. This helps prevent unauthorized access from other processes and users on the database server. For more information, see https://blueprints.launchpad.net/tripleo/+spec/tls-via-certmonger

You can optionally restrict the set of SSL ciphers used for the encrypted connection. See ciphers for a list of ciphers and the syntax for specifying the cipher string:

ssl-cipher = 'cipher:list'

When TLSEverywhere is enabled, Red Hat OpenStack Platform 13 is able to pass the cipher list in mysqld’s configuration file: /var/lib/config-data/puppet-generated/mysql/etc/my.cnf.d/galera.cnf. By default, TLSv1.0 ciphers are disabled. If required, you can override the default with the cipher_list and gcomm_cipher Heat parameters. For more information, see https://review.openstack.org/#/c/566509.