Red Hat Training

A Red Hat training course is available for Red Hat Ceph Storage

Chapter 4. Mounting and Unmounting Ceph File Systems

There are two ways to temporarily mount a Ceph File System:

On details on mounting Ceph File Systems permanently, see Section 4.4, “Mounting Ceph File Systems Permanently in /etc/fstab”.

Before mounting a CephFS client, create a client keyring with capabilities that specifies client access rights and permissions. See Section 4.1, “Client Authentication” for details.

4.1. Client Authentication

To restrict the Ceph File System clients to the lowest possible level of authority needed, use Ceph authentication capabilities.

CephFS supports the following restrictions:

Path Restriction

By default, clients are not restricted in what paths they are allowed to mount. Further, when clients mount a subdirectory, for example, /home/<user>, the MDS does not by default verify that subsequent operations are locked within that directory.

To restrict clients to only mount and work within a certain directory, use path-based MDS authentication capabilities. For example, to restrict the MDS daemon to write metadata only to a particular directory, specify that directory while creating the client capabilities: 

ceph auth get-or-create client.<client-name/id> mon 'allow r' mds 'allow r, allow rw path=<directory>' osd 'allow rw pool=data'

Example

The following example command restricts the MDS to write metadata only to the /home/cephfs/ directory. Also, it restricts the CephFS client to perform read and write operations only within the data pool: 

$ ceph auth get-or-create client.1 mon 'allow r' mds 'allow r, allow rw path=/home/cephfs' osd 'allow rw pool=data'
[client.1]
    key = AQACNoZXhrzqIRAABPKHTach4x03JeNadeQ9Uw==

To view the created key: 

$ ceph auth get client.1
exported keyring for client.1
[client.1]
    key = AQACNoZXhrzqIRAABPKHTach4x03JeNadeQ9Uw==
    caps mds = "allow r, allow rw path=/home/cephfs"
    caps mon = "allow r"
    caps osd = "allow rw pool=data"

Path restriction using the authentication capabilities is the most common way to restrict clients. See the User Management chapter in the Administration Guide for details on authentication capabilities.

When a client has capabilities that restrict it to a path, use the -r option with the ceph-fuse command so that the client will treat that path as its root: 

ceph-fuse -n client.<client-name/id> --keyring=<path_to_keyring> <mount-point> -r <directory>

Example

To instruct the client with ID 1 to treat the /home/cephfs/ directory as its root: 

# ceph-fuse -n client.1 --keyring=/etc/ceph/client.1.keyring /mnt/cephfs -r /home/cephf
Note

If you use the default location and name of the client keyring, that is /etc/ceph/ceph.client.<client-ID>.keyring, you do not have to use the --keyring option.

OSD restriction

To prevent clients from writing to or reading data from pools other than those in use for the Ceph File System, set an OSD authentication capability that restricts access to the CephFS data pools: 

client.<client-name/id>
    key: <key>
    caps: [mds] allow rw
    caps: [mon] allow r
    caps: [osd] allow rw pool=<pool-name>

To restrict clients from writing data, use r instead of rw: 

client.<client-name/id>
    key: <key>
    caps: [mds] allow rw
    caps: [mon] allow r
    caps: [osd] allow r pool=<pool-name>

This does not affect the ability of the clients to update file system metadata for files it has read access to, but it prevents them from persistently writing data in a way that would be visible to other clients.

Example:

To restrict client with id 1 to have read and write access to pool data and read access to pool stack: 

client.1
    key: AQAz7EVWygILFRAAdIcuJ12opU/JKyfFmxhuaw==
    caps: [mds] allow rw
    caps: [mon] allow r
    caps: [osd] allow rw pool=data, allow r pool=stack

See the User Management chapter in the Administration Guide for details.

Layout Modification Restriction

To prevent clients from modifying the data pool used for files or directories, use the p modifier in MDS authentication capabilities.

Example

In the following snippet client.0 can modify the pool used for files, but client.1 cannot: 

client.0
    key: AQAz7EVWygILFRAAdIcuJ12opU/JKyfFmxhuaw==
    caps: [mds] allow rwp
    caps: [mon] allow r
    caps: [osd] allow rw pool=data

client.1
    key: AQAz7EVWygILFRAAdIcuJ12opU/JKyfFmxhuaw==
    caps: [mds] allow rw
    caps: [mon] allow r
    caps: [osd] allow rw pool=data

4.2. Mounting Ceph File Systems as Kernel Clients

To mount a Ceph File System as a kernel client, use the mount utility.

  1. On the client node, enable the Red Hat Ceph Storage 2 Tools repository. For details, see the Enabling Ceph Repositories section in the Installation Guide for Red Hat Enterprise Linux or the Enabling Ceph Repositories section in the Installation Guide for Ubuntu.

  2. Ensure that the ceph-common package is installed on the client and if not, install it:

    • On Red Hat Enterprise Linux: 

      # yum install ceph-common

    • On Ubuntu: 

      $ sudo apt-get install ceph-common

  3. Mount the Ceph File System. To specify multiple monitor addresses, either separate them with commas in the mount command, or configure a DNS server so that a single host name resolves to multiple IP addresses and pass that host name to the mount command. For details on setting DNS servers see the DNS Servers chapter in the Networking Guide for Red Hat Enterprise Linux 7. 

    mount -t ceph <monitor1-host-name>:6789,<monitor2-host-name>:6789,<monitor3-host-name>:6789:/ <mount-point>

    Example: 

    # mount -t ceph mon1:6789,mon2:6789,mon3:6789:/ /mnt/cephfs

    To mount a Ceph File System with the cephx authentication enabled, specify a user name and a secret file: 

    mount -t ceph <monitor-hostname>:6789:/ <mount-point> -o name=<username>, secretfile=<secret-file>

    Example 

    # mount -t ceph mon1:6789:/ /mnt/cephfs -o name=user,secretfile=/etc/ceph/user.secret

For details on cephx, see the User Management chapter in the Administration Guide.

For more information about mount, see the mount(8) manual page.

4.3. Mounting Ceph File Systems in User Space (FUSE)

To mount a Ceph File System as a FUSE client:

  1. On the client node, enable the Red Hat Ceph Storage 2 Tools repository. For details, see the Enabling Ceph Repositories section in the Installation Guide for Red Hat Enterprise Linux or the Enabling Ceph Repositories section in the Installation Guide for Ubuntu.

  2. Ensure that the ceph-common and ceph-fuse packages are installed on the client and if not, install them.

    • On Red Hat Enterprise Linux: 

      # yum install ceph-common ceph-fuse

    • On Ubuntu: 

      $ sudo apt-get ceph-common ceph-fuse

  3. Copy the Ceph configuration file from the monitor host to the /etc/ceph/ directory on the client host: 

    scp root@<mon-host>:/etc/ceph/ceph.conf /etc/ceph/ceph.conf

    Replace <mon-host> with the monitor host name or IP, for example: 

    # scp root@192.168.0.1:/etc/ceph/ceph.conf /etc/ceph/ceph.conf

  4. On the administration or monitor host, create the client user with correct authentication capabilities and output the user keyring to a file: 

    ceph auth get-or-create client.<client-name/id> mon 'allow r' mds 'allow r, allow rw path=<directory>' osd 'allow rw pool=<pool>' -o <file_name>

    Specify the client name or ID, the CephFS working directory, pool and the output file. For example: 

    $ ceph auth get-or-create client.1 mon 'allow r' mds 'allow r, allow rw path=/' osd 'allow rw pool=data' -o ceph.client.1.keyring
[client.1]
	key = AQACNoZXhrzqIRAABPKHTach4x03JeNadeQ9Uw==

  5. Copy the client keyring from the monitor host to the /etc/ceph/ directory on the client host: 

    scp root@<mon-host>:/ceph.client.1.keyring /etc/ceph/ceph.client.1.keyring

    Replace <mon-host> with the monitor host name or IP, for example: 

    # scp root@192.168.0.1:/ceph.client.1.keyring /etc/ceph/ceph.client.1.keyring

  6. Ensure that the Ceph configuration file and the keyring have correct permissions: 

    # chmod 644 /etc/ceph/ceph.conf
# chmod 644 /etc/ceph/ceph.client.1.keyring

  7. Create a directory to serve as a mount point. Note that the mount point must be within what is permitted by the client capabilities by the path option: 

    $ mkdir <mountpoint>

    For example: 

    $ mkdir /mnt/mycephfs

  8. Use the ceph-fuse utility to mount the Ceph File System: 

    ceph-fuse -n client.<client-name> -m <monitor1-host-name>:6789, <monitor2-host-name>:6789, <monitor3-host-name>:6789 <mountpoint>

    For example: 

    # ceph-fuse -n client.1 -m mon1:6789, mon2:6789, mon3:6789 /mnt/mycephfs

    If you do not use the default name and location of the user keyring, that is /etc/ceph/ceph.client.<client-name/id>.keyring, use the --keyring option to specify the path to the user keyring, for example: 

    # ceph-fuse -n client.1 -m mon1:6789, mon2:6789, mon3:6789 --keyring=/etc/ceph/client1.keyring /mnt/mycephfs

For more information about ceph-fuse see the ceph-fuse(8) manual page.

4.4. Mounting Ceph File Systems Permanently in /etc/fstab

To automatically mount Ceph File Systems on startup, add them to the /etc/fstab file. The form of the entry depends on how the Ceph File System is mounted.

In all cases, use the _netdev option. This option ensures that the file system is mounted after the networking subsystem to prevent networking issues.

Ceph File System mounted as a kernel client

#DEVICE               	   PATH           TYPE     OPTIONS
<mon1-hostanme>:<port>:/,  <mountpoint>   ceph     [name=username
<mon1-hostanme>:<port>:/,        	           ,secret=secretkey|
<mon1-hostanme>:<port>:/		           secretfile=
						   path_to_secretfile],
						   [<mount.options>]

Example 

mon1:6789:/,    	 /mnt/cephfs   ceph    	name=admin,
mon2:6789:/,					secretfile=
mon3:6789:/					/etc/ceph/secret.key,
					    	_netdev,
					   	noatime 0 0
Important

The name and secret or secretfile options are mandatory when Ceph authentication is enabled.

Ceph File System mounted as a FUSE client

#DEVICE                                  PATH          TYPE      OPTIONS
id=<user-ID>[,conf=<configuration_file>] <mount-point> fuse.ceph _netdev,
                                                                 defaults
								 0 0

Examples 

id=client1  				/mnt/ceph  fuse.ceph  	_netdev,
								defaults
								0 0
id=myuser,conf=/etc/ceph/ceph.conf  	/mnt/ceph2  fuse.ceph 	_netdev,
								defaults
								0 0

The DEVICE field is a comma-delimited list of options to pass to the command line. Ensure to use the ID (for example, admin, not client.admin). You can pass any valid ceph-fuse option to the command line this way.

Important

The util-linux package shipped with Red Hat Enterprise Linux 7.2 does not support mounting CephFS FUSE clients in /etc/fstab. Red Hat Enterprise Linux 7.3 includes a new version of util-linux that supports mounting CephFS FUSE clients permanently.

4.5. Unmounting Ceph File Systems

Unmounting Ceph File Systems mounted as kernel clients

To unmount a Ceph File System mounted as a kernel client: 

umount <mount-point>

Example 

# umount /mnt/cephfs

See the umount(8) manual page for details.

Unmounting Ceph File Systems mounted as FUSE

To unmount a Ceph File System mounted in FUSE: 

fusermount -u <mount-point>

Example 

# fusermount -u /mnt/cephfs

See the ceph-fuse(8) manual page for details.