Red Hat Training

A Red Hat training course is available for Red Hat Ceph Storage

Chapter 4. Cephx Configuration Reference

The cephx protocol is enabled by default. Cryptographic authentication has some computational costs, though they are generally quite low. If the network environment connecting a client and server hosts is very safe and you cannot afford authentication, you can disable it. However, Red Hat recommends using authentication.

Note

If you disable authentication, you are at risk of a man-in-the-middle attack altering client and server messages, which could lead to significant security issues.

4.1. Manual

When you deploy a cluster manually, you have to bootstrap the monitor manually and create the client.admin user and keyring. To deploy Ceph manually, see our Knowledgebase article. The steps for monitor bootstrapping are the logical steps you must perform when using third party deployment tools like Chef, Puppet, Juju, and so on.

4.2. Enabling and Disabling Cephx

Enabling Cephx requires that you have deployed keys for your monitors and OSDs. If you are simply toggling Cephx on / off, you do not have to repeat the bootstrapping procedures.

4.2.1. Enabling Cephx

When cephx is enabled, Ceph will look for the keyring in the default search path, which includes /etc/ceph/$cluster.$name.keyring. You can override this location by adding a keyring option in the [global] section of the Ceph configuration file, but this is not recommended.

Execute the following procedures to enable cephx on a cluster with authentication disabled. If you or your deployment utility have already generated the keys, you may skip the steps related to generating keys.

  1. Create a client.admin key, and save a copy of the key for your client host: 

    ceph auth get-or-create client.admin mon 'allow *' osd 'allow *' -o /etc/ceph/ceph.client.admin.keyring
    Warning

    This will erase the contents of any existing /etc/ceph/client.admin.keyring file. Do not perform this step if a deployment tool has already done it for you.

  2. Create a keyring for the monitor cluster and generate a monitor secret key: 

    ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'

  3. Copy the monitor keyring into a ceph.mon.keyring file in every monitor mon data directory. For example, to copy it to mon.a in cluster ceph, use the following: 

    cp /tmp/ceph.mon.keyring /var/lib/ceph/mon/ceph-a/keyring

  4. Generate a secret key for every OSD, where {$id} is the OSD number: 

    ceph auth get-or-create osd.{$id} mon 'allow rwx' osd 'allow *' -o /var/lib/ceph/osd/ceph-{$id}/keyring

  5. By default the cephx authentication protocol is enabled.

    Note

    If the cephx authentication protocol was disabled previously by setting the authentication options to none, then by removing the following lines under the [global] section in the Ceph configuration file (/etc/ceph/ceph.conf) will reenable the cephx authentication protocol: 

    auth_cluster_required = none
auth_service_required = none
auth_client_required = none

  6. Start or restart the Ceph cluster.

    Important

    Enabling cephx requires downtime because the cluster needs to be completely restarted, or it needs to be shut down and then started while client I/O is disabled.

    These flags need to be set before restarting or shutting down the storage cluster: 

    # ceph osd set noout
# ceph osd set norecover
# ceph osd set norebalance
# ceph osd set nobackfill
# ceph osd set nodown
# ceph osd set pause

    Once cephx is enabled and all PGs are active and clean, unset the flags: 

    # ceph osd unset noout
# ceph osd unset norecover
# ceph osd unset norebalance
# ceph osd unset nobackfill
# ceph osd unset nodown
# ceph osd unset pause

4.2.2. Disabling Cephx

The following procedure describes how to disable Cephx. If your cluster environment is relatively safe, you can offset the computation expense of running authentication. Red Hat recommends enabling authentication. However, it may be easier during setup or troubleshooting to temporarily disable authentication.

  1. Disable cephx authentication by setting the following options in the [global] section of the Ceph configuration file: 

    auth_cluster_required = none
auth_service_required = none
auth_client_required = none
  2. Start or restart the Ceph cluster.

4.3. Configuration Settings

4.3.1. Enablement

auth_cluster_required
Description
If enabled, the Red Hat Ceph Storage cluster daemons (that is, ceph-mon and ceph-osd) must authenticate with each other. Valid settings are cephx or none.
Type
String
Required
No
Default
cephx.
auth_service_required
Description
If enabled, the Red Hat Ceph Storage cluster daemons require Ceph clients to authenticate with the Red Hat Ceph Storage cluster in order to access Ceph services. Valid settings are cephx or none.
Type
String
Required
No
Default
cephx.
auth_client_required
Description
If enabled, the Ceph client requires the Red Hat Ceph Storage cluster to authenticate with the Ceph client. Valid settings are cephx or none.
Type
String
Required
No
Default
cephx.

4.3.2. Keys

When you run Ceph with authentication enabled, the ceph administrative commands and Ceph clients require authentication keys to access the Ceph storage cluster.

The most common way to provide these keys to the ceph administrative commands and clients is to include a Ceph keyring under the /etc/ceph/ directory. The file name is usually ceph.client.admin.keyring or $cluster.client.admin.keyring. If you include the keyring under the /etc/ceph/ directory, you do not need to specify a keyring entry in the Ceph configuration file.

Red Hat recommends copying the Red Hat Ceph Storage cluster keyring file to nodes where you will run administrative commands, because it contains the client.admin key. To do so, execute the following command as root: 

# scp <user>@<hostname>:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring

Replace <user> with the user name used on the host with the client.admin key and <hostname> with the host name of that host.

Note

Ensure the ceph.keyring file has appropriate permissions set on the client machine.

You can specify the key itself in the Ceph configuration file using the key setting, which is not recommended, or a path to a key file using the keyfile setting.

keyring
Description
The path to the keyring file.
Type
String
Required
No
Default
/etc/ceph/$cluster.$name.keyring,/etc/ceph/$cluster.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin
keyfile
Description
The path to a key file (that is. a file containing only the key).
Type
String
Required
No
Default
None
key
Description
The key (that is, the text string of the key itself). Not recommended.
Type
String
Required
No
Default
None

4.3.3. Daemon Keyrings

Administrative users or deployment tools might generate daemon keyrings in the same way as generating user keyrings. By default, Ceph stores daemons keyrings inside their data directory. The default keyring locations, and the capabilities necessary for the daemon to function, are shown below.

ceph-mon
Location
$mon_data/keyring
Capabilities
mon 'allow *'
ceph-osd
Location
$osd_data/keyring
Capabilities
mon 'allow profile osd' osd 'allow *'
radosgw
Location
$rgw_data/keyring
Capabilities
mon 'allow rwx' osd 'allow rwx'
Note

The monitor keyring (that is mon.) contains a key but no capabilities, and is not part of the cluster auth database.

The daemon data directory locations default to directories of the form: 

/var/lib/ceph/$type/$cluster-$id

For example, osd.12 is: 

/var/lib/ceph/osd/ceph-12

You can override these locations, but it is not recommended.

4.3.4. Signatures

Red Hat recommends that Ceph authenticate all ongoing messages between the entities using the session key set up for that initial authentication.

Like other parts of Ceph authentication, Ceph provides fine-grained control so you can enable or disable signatures for service messages between the client and Ceph, and you can enable or disable signatures for messages between Ceph daemons.

cephx_require_signatures
Description
If set to true, Ceph requires signatures on all message traffic between the Ceph client and the Red Hat Ceph Storage cluster, and between daemons comprising the Red Hat Ceph Storage cluster.
Type
Boolean
Required
No
Default
false
cephx_cluster_require_signatures
Description
If set to true, Ceph requires signatures on all message traffic between Ceph daemons comprising the Red Hat Ceph Storage cluster.
Type
Boolean
Required
No
Default
false
cephx_service_require_signatures
Description
If set to true, Ceph requires signatures on all message traffic between Ceph clients and the Red Hat Ceph Storage cluster.
Type
Boolean
Required
No
Default
false
cephx_sign_messages
Description
If the Ceph version supports message signing, Ceph will sign all messages so they cannot be spoofed.
Type
Boolean
Default
true
Note

Ceph kernel modules do not support signatures yet.

4.3.5. Time to Live

auth_service_ticket_ttl
Description
When the Red Hat Ceph Storage cluster sends a Ceph client a ticket for authentication, the cluster assigns the ticket a time to live.
Type
Double
Default
60*60