25.6. Configuring the Object Store

This section provides instructions on how to configure Object Store in your storage environment.

Warning

When you install Red Hat Gluster Storage 3.1, the /etc/swift directory would contain both *.conf extension and *.conf-gluster files. You must delete the *.conf files and create new configuration files based on *.conf-gluster template. Otherwise, inappropriate python packages will be loaded and the component may not work as expected.
If you are upgrading to Red Hat Gluster Storage 3.1, the older configuration files will be retained and new configuration files will be created with .rpmnew extension. You must ensure to delete .conf files and folders (account-server, container-server, and object-server) for better understanding of the loaded configuration.

25.6.1. Configuring a Proxy Server

Create a new configuration file /etc/swift/proxy-server.conf by referencing the template file available at /etc/swift/proxy-server.conf-gluster.

25.6.1.1. Configuring a Proxy Server for HTTPS

By default, proxy server only handles HTTP requests. To configure the proxy server to process HTTPS requests, perform the following steps:
  1. Create self-signed cert for SSL using the following commands:
    # cd /etc/swift
    # openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
  2. Add the following lines to /etc/swift/proxy-server.conf under [DEFAULT]
    bind_port = 443
     cert_file = /etc/swift/cert.crt
     key_file = /etc/swift/cert.key

Important

When Object Storage is deployed on two or more machines, not all nodes in your trusted storage pool are used. Installing a load balancer enables you to utilize all the nodes in your trusted storage pool by distributing the proxy server requests equally to all storage nodes.
Memcached allows nodes' states to be shared across multiple proxy servers. Edit the memcache_servers configuration option in the proxy-server.conf and list all memcached servers.
Following is an example listing the memcached servers in the proxy-server.conf file.
[filter:cache]
use = egg:swift#memcache
memcache_servers = 192.168.1.20:11211,192.168.1.21:11211,192.168.1.22:11211
The port number on which the memcached server is listening is 11211. You must ensure to use the same sequence for all configuration files.

25.6.2. Configuring the Authentication Service

This section provides information on configuring Keystone, GSwauth, and TempAuth authentication services.

25.6.2.1. Integrating with the Keystone Authentication Service

  • To configure Keystone, add authtoken and keystoneauth to /etc/swift/proxy-server.conf pipeline as shown below:
    [pipeline:main]
    pipeline = catch_errors healthcheck proxy-logging cache authtoken keystoneauth proxy-logging proxy-server
  • Add the following sections to /etc/swift/proxy-server.conf file by referencing the example below as a guideline. You must substitute the values according to your setup:
    [filter:authtoken]
    paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
    signing_dir = /etc/swift
    auth_host = keystone.server.com
    auth_port = 35357
    auth_protocol = http
    auth_uri = http://keystone.server.com:5000
    # if its defined
    admin_tenant_name = services
    admin_user = swift
    admin_password = adminpassword
    delay_auth_decision = 1
    
    [filter:keystoneauth]
    use = egg:swift#keystoneauth
    operator_roles = admin, SwiftOperator
    is_admin = true
    cache = swift.cache
Verify the Integrated Setup

Verify that the Red Hat Gluster Storage Object Store has been configured successfully by running the following command:

$ swift -V 2 -A http://keystone.server.com:5000/v2.0 -U tenant_name:user -K password stat

25.6.2.2. Integrating with the GSwauth Authentication Service

Integrating GSwauth

Perform the following steps to integrate GSwauth:

  1. Create and start a Red Hat Gluster Storage volume to store metadata.
    # gluster volume create NEW-VOLNAME NEW-BRICK
    # gluster volume start NEW-VOLNAME
    For example:
    # gluster volume create gsmetadata server1:/exp1 
    # gluster volume start gsmetadata
  2. Run gluster-swift-gen-builders tool with all the volumes to be accessed using the Swift client including gsmetadata volume:
    # gluster-swift-gen-builders gsmetadata other volumes
  3. Edit the /etc/swift/proxy-server.conf pipeline as shown below:
    [pipeline:main]
    pipeline = catch_errors cache gswauth proxy-server
  4. Add the following section to /etc/swift/proxy-server.conf file by referencing the example below as a guideline. You must substitute the values according to your setup.
    [filter:gswauth]
    use = egg:gluster_swift#gswauth
    set log_name = gswauth
    super_admin_key = gswauthkey
    metadata_volume = gsmetadata
    auth_type = sha1
    auth_type_salt = swauthsalt

    Important

    You must ensure to secure the proxy-server.conf file and the super_admin_key option to prevent unprivileged access.
  5. Restart the proxy server by running the following command:
    # swift-init proxy restart
Advanced Options:

You can set the following advanced options for GSwauth WSGI filter:

  • default-swift-cluster: The default storage-URL for the newly created accounts. When you attempt to authenticate for the first time, the access token and the storage-URL where data for the given account is stored will be returned.
  • token_life: The set default token life. The default value is 86400 (24 hours).
  • max_token_life: The maximum token life. You can set a token lifetime when requesting a new token with header x-auth-token-lifetime. If the passed in value is greater than the max_token_life, then the max_token_life value will be used.
GSwauth Common Options of CLI Tools

GSwauth provides CLI tools to facilitate managing accounts and users. All tools have some options in common:

  • -A, --admin-url: The URL to the auth. The default URL is http://127.0.0.1:8080/auth/.
  • -U, --admin-user: The user with administrator rights to perform action. The default user role is .super_admin.
  • -K, --admin-key: The key for the user with administrator rights to perform the action. There is no default value.
Preparing Red Hat Gluster Storage Volumes to Save Metadata

Prepare the Red Hat Gluster Storage volume for gswauth to save its metadata by running the following command:

# gswauth-prep [option]
For example:
# gswauth-prep -A http://10.20.30.40:8080/auth/ -K gswauthkey
25.6.2.2.1. Managing Account Services in GSwauth
Creating Accounts

Create an account for GSwauth. This account is mapped to a Red Hat Gluster Storage volume.

# gswauth-add-account [option] <account_name>
For example:
# gswauth-add-account -K gswauthkey <account_name>
Deleting an Account

You must ensure that all users pertaining to this account must be deleted before deleting the account. To delete an account:

# gswauth-delete-account [option] <account_name>
For example:
# gswauth-delete-account -K gswauthkey test
Setting the Account Service

Sets a service URL for an account. User with reseller admin role only can set the service URL. This command can be used to change the default storage URL for a given account. All accounts will have the same storage-URL as default value, which is set using default-swift-cluster option.

# gswauth-set-account-service [options] <account> <service> <name> <value>
For example:
# gswauth-set-account-service -K gswauthkey test storage local http://newhost:8080/v1/AUTH_test
25.6.2.2.2. Managing User Services in GSwauth
User Roles

The following user roles are supported in GSwauth:

  • A regular user has no rights. Users must be given both read and write privileges using Swift ACLs.
  • The admin user is a super-user at the account level. This user can create and delete users for that account. These members will have both write and read privileges to all stored objects in that account.
  • The reseller admin user is a super-user at the cluster level. This user can create and delete accounts and users and has read and write privileges to all accounts under that cluster.
  • GSwauth maintains its own swift account to store all of its metadata on accounts and users. The .super_admin role provides access to GSwauth own swift account and has all privileges to act on any other account or user.
User Access Matrix

The following table provides user access right information.

Table 25.2. User Access Matrix

Role/Group get list of accounts get Acccount Details Create Account Delete Account Get User Details Create admin user Create reseller_admin user Create regular user Delete admin user
.super_admin (username) X X X X X X X X X
.reseller_admin (group) X X X X X X   X X
.admin (group)   X    X X   X X
regular user (type)          
Creating Users

You can create an user for an account that does not exist. The account will be created before creating the user.

You must add -r flag to create a reseller admin user and -a flag to create an admin user. To change the password or role of the user, you can run the same command with the new option.
# gswauth-add-user [option] <account_name> <user> <password>
For example
# gswauth-add-user -K gswauthkey -a test ana anapwd
Deleting a User

Delete a user by running the following command:

gswauth-delete-user [option] <account_name> <user>
For example
gwauth-delete-user -K gswauthkey test ana
Authenticating a User with the Swift Client

There are two methods to access data using the Swift client. The first and simple method is by providing the user name and password everytime. The swift client will acquire the token from gswauth.

For example:
$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:ana -K anapwd upload container1 README.md
The second method is a two-step process, first you must authenticate with a username and password to obtain a token and the storage URL. Then, you can make the object requests to the storage URL with the given token.
It is important to remember that tokens expires, so the authentication process needs to be repeated very often.
Authenticate a user with the cURL command:
curl -v -H 'X-Storage-User: test:ana' -H 'X-Storage-Pass: anapwd' -k http://localhost:8080/auth/v1.0
...
< X-Auth-Token: AUTH_tk7e68ef4698f14c7f95af07ab7b298610
< X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test
...
Now, you use the given token and storage URL to access the object-storage using the Swift client:
$ swift --os-auth-token=AUTH_tk7e68ef4698f14c7f95af07ab7b298610 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test upload container1 README.md
README.md
bash-4.2$ 
bash-4.2$ swift --os-auth-token=AUTH_tk7e68ef4698f14c7f95af07ab7b298610 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test list container1
README.md

Important

Reseller admins must always use the second method to acquire a token to get access to other accounts other than his own. The first method of using the username and password will give them access only to their own accounts.
25.6.2.2.3. Managing Accounts and Users Information
Obtaining Accounts and User Information

You can obtain the accounts and users information including stored password.

# gswauth-list [options] [account] [user]
For example:
# gswauth-list -K gswauthkey test ana
+----------+
|  Groups  |
+----------+
| test:ana |
|   test   |
|  .admin  |
+----------+
  • If [account] and [user] are omitted, all the accounts will be listed.
  • If [account] is included but not [user], a list of users within that account will be listed.
  • If [account] and [user] are included, a list of groups that the user belongs to will be listed.
  • If the [user] is .groups, the active groups for that account will be listed.
The default output format is in tabular format. Adding -p option provides the output in plain text format, -j provides the output in JSON format.
Changing User Password

You can change the password of the user, account administrator, and reseller_admin roles.

  • Change the password of a regular user by running the following command:
    # gswauth-add-user -U account1:user1 -K old_passwd account1 user1 new_passwd
  • Change the password of an account administrator by running the following command:
    # gswauth-add-user -U account1:admin -K old_passwd -a account1 admin new_passwd
  • Change the password of the reseller_admin by running the following command:
    # gswauth-add-user -U account1:radmin -K old_passwd -r account1 radmin new_passwd
Cleaning Up Expired Tokens

Users with .super_admin role can delete the expired tokens.

You also have the option to provide the expected life of tokens, delete all tokens or delete all tokens for a given account.
# gswauth-cleanup-tokens [options]
For example
# gswauth-cleanup-tokens -K gswauthkey --purge test
The tokens will be deleted on the disk but it would still persist in memcached.
You can add the following options while cleaning up the tokens:
  • -t, --token-life: The expected life of tokens. The token objects modified before the give number of seconds will be checked for expiration (default: 86400).
  • --purge: Purges all the tokens for a given account whether the tokens have expired or not.
  • --purge-all: Purges all the tokens for all the accounts and users whether the tokens have expired or not.

25.6.2.3. Integrating with the TempAuth Authentication Service

Warning

TempAuth authentication service must only be used in test deployments and not for production.
TempAuth is automatically installed when you install Red Hat Gluster Storage. TempAuth stores user and password information as cleartext in a single proxy-server.conf file. In your /etc/swift/proxy-server.conf file, enable TempAuth in pipeline and add user information in TempAuth section by referencing the below example.
[pipeline:main]
pipeline = catch_errors healthcheck proxy-logging cache tempauth proxy-logging proxy-server

[filter:tempauth]
use = egg:swift#tempauth
user_admin_admin = admin.admin.reseller_admin
user_test_tester = testing .admin
user_test_tester2 = testing2
You can add users to the account in the following format:
user_accountname_username = password [.admin]
Here the accountname is the Red Hat Gluster Storage volume used to store objects.
You must restart the Object Store services for the configuration changes to take effect. For information on restarting the services, see Section 25.6.9, “Starting and Stopping Server”.

25.6.3. Configuring Object Servers

Create a new configuration file /etc/swift/object.server.conf by referencing the template file available at /etc/swift/object-server.conf-gluster.

25.6.4. Configuring Container Servers

Create a new configuration file /etc/swift/container-server.conf by referencing the template file available at /etc/swift/container-server.conf-gluster.

25.6.5. Configuring Account Servers

Create a new configuration file /etc/swift/account-server.conf by referencing the template file available at /etc/swift/account-server.conf-gluster.

25.6.6. Configuring Swift Object and Container Constraints

Create a new configuration file /etc/swift/swift.conf by referencing the template file available at /etc/swift/swift.conf-gluster.

25.6.7. Configuring Object Expiration

The Object Expiration feature allows you to schedule automatic deletion of objects that are stored in the Red Hat Gluster Storage volume. You can use the object expiration feature to specify a lifetime for specific objects in the volume; when the lifetime of an object expires, the object store would automatically quit serving that object and would shortly thereafter remove the object from the Red Hat Gluster Storage volume. For example, you might upload logs periodically to the volume, and you might need to retain those logs for only a specific amount of time.
The client uses the X-Delete-At or X-Delete-After headers during an object PUT or POST and the Red Hat Gluster Storage volume would automatically quit serving that object.

Note

Expired objects appear in container listings until they are deleted by the object-expirer daemon. This is an expected behavior.
A DELETE object request on an expired object would delete the object from Red Hat Gluster Storage volume (if it is yet to be deleted by the object expirer daemon). However, the client would get a 404 (Not Found) status in return. This is also an expected behavior.

25.6.7.1. Setting Up Object Expiration

Object expirer uses a separate account (a Red Hat Gluster Storage volume) named gsexpiring for managing object expiration. Hence, you must create a Red Hat Gluster Storage volume and name it as gsexpiring.
Create a new configuration file /etc/swift/object.expirer.conf by referencing the template file available at /etc/swift/object-expirer.conf-gluster.

25.6.7.2. Using Object Expiration

When you use the X-Delete-At or X-Delete-After headers during an object PUT or POST, the object is scheduled for deletion. The Red Hat Gluster Storage volume would automatically quit serving that object at the specified time and will shortly thereafter remove the object from the Red Hat Gluster Storage volume.
Use PUT operation while uploading a new object. To assign expiration headers to existing objects, use the POST operation.
X-Delete-At header

The X-Delete-At header requires a UNIX epoch timestamp, in integer form. For example, 1418884120 represents Thu, 18 Dec 2014 06:27:31 GMT. By setting the header to a specific epoch time, you indicate when you want the object to expire, not be served, and be deleted completely from the Red Hat Gluster Storage volume. The current time in Epoch notation can be found by running this command:

$ date +%s

  • Set the object expiry time during an object PUT with X-Delete-At header using cURL:
    curl -v -X PUT -H 'X-Delete-At: 1392013619' http://127.0.0.1:8080/v1/AUTH_test/container1/object1 -T ./localfile
    Set the object expiry time during an object PUT with X-Delete-At header using swift client:
    swift --os-auth-token=AUTH_tk99a39aecc3dd4f80b2b1e801d00df846 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test upload container1 ./localfile --header 'X-Delete-At: 1392013619'
X-Delete-After

The X-Delete-After header takes an integer number of seconds that represents the amount of time from now when you want the object to be deleted.

  • Set the object expiry time with an object PUT with X-Delete-After header using cURL:
    curl -v -X PUT -H 'X-Delete-After: 3600' http://127.0.0.1:8080/v1/AUTH_test/container1/object1 -T ./localfile
    Set the object expiry time with an object PUT with X-Delete-At header using swift client:
    swift --os-auth-token=AUTH_tk99a39aecc3dd4f80b2b1e801d00df846 --os-storage-url=http://127.0.0.1:8080/v1/AUTH_test upload container1 ./localfile --header 'X-Delete-After: 3600'

25.6.7.3. Running Object Expirer Service

The object-expirer service runs once in every 300 seconds, by default. You can modify the duration by configuring interval option in /etc/swift/object-expirer.conf file. For every pass it makes, it queries the gsexpiring account for tracker objects. Based on the timestamp and path present in the name of tracker objects, object-expirer deletes the actual object and the corresponding tracker object.
To start the object-expirer service:
# swift-init object-expirer start
To run the object-expirer once:
# swift-object-expirer -o -v /etc/swift/object-expirer.conf

25.6.8. Exporting the Red Hat Gluster Storage Volumes

After creating configuration files, you must now add configuration details for the system to identify the Red Hat Gluster Storage volumes to be accessible as Object Store. These configuration details are added to the ring files. The ring files provide the list of Red Hat Gluster Storage volumes to be accessible using the object storage interface to the Swift on File component.
Create the ring files for the current configurations by running the following command:
# cd /etc/swift
# gluster-swift-gen-builders VOLUME [VOLUME...]
For example,
# cd /etc/swift
# gluster-swift-gen-builders testvol1 testvol2 testvol3
Here testvol1, testvol2, and testvol3 are the Red Hat Gluster Storage volumes which will be mounted locally under the directory mentioned in the object, container, and account configuration files (default value is /mnt/gluster-object). The default value can be changed to a different path by changing the devices configurable option across all account, container, and object configuration files. The path must contain Red Hat Gluster Storage volumes mounted under directories having the same names as volume names. For example, if devices option is set to /home, it is expected that the volume named testvol1 be mounted at /home/testvol1.
Note that all the volumes required to be accessed using the Swift interface must be passed to the gluster-swift-gen-builders tool even if it was previously added. The gluster-swift-gen-builders tool creates new ring files every time it runs successfully.
To remove a VOLUME, run gluster-swift-gen-builders only with the volumes which are required to be accessed using the Swift interface.
For example, to remove the testvol2 volume, run the following command:
# gluster-swift-gen-builders testvol1 testvol3
You must restart the Object Store services after creating the new ring files.

25.6.9. Starting and Stopping Server

You must start or restart the server manually whenever you update or modify the configuration files. These processes must be owned and run by the root user.
  • To start the server, run the following command:
    # swift-init main start
  • To stop the server, run the following command:
    # swift-init main stop
  • To restart the server, run the following command:
    # swift-init main restart