Red Hat Training

A Red Hat training course is available for Red Hat Ceph Storage

Chapter 7. Object Gateway Swift API

Ceph supports a RESTful API that is compatible with the basic data access model of the Swift API.

7.1. Features Support

The following table describes the support status for current Swift functional features:

FeatureStatusRemarks

Authentication

Supported

 

Get Account Metadata

Supported

No custom metadata

Swift ACLs

Supported

Supports a subset of Swift ACLs

List Containers

Supported

 

Delete Container

Supported

 

Create Container

Supported

 

Get Container Metadata

Supported

 

Update Container Metadata

Supported

 

Delete Container Metadata

Supported

 

List Objects

Supported

 

Static Website

Not Supported

 

Create/Update an Object

Supported

 

Create Large Object

Supported

 

Delete Object

Supported

 

Get Object

Supported

 

Copy Object

Supported

 

Get Object Metadata

Supported

 

Add/Update Object Metadata

Supported

 

Temp URL Operations

Supported

 

Expiring Objects

Not Supported

 

Object Versioning

Not Supported

 

CORS

Not Supported

 

7.2. Authentication

Swift API requests that require authentication must contain an X-Storage-Token authentication token in the request header. The token may be retrieved from Ceph Object Gateway, or from another authenticator. To obtain a token from Ceph Object Gateway, you must create a user. For example:

sudo radosgw-admin user create --uid="{username}" --display-name="{Display Name}"

7.2.1. Authentication GET

To authenticate a user, make a request containing an X-Auth-User and a X-Auth-Key in the header.

7.2.1.1. Syntax

GET /auth HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-User: johndoe
X-Auth-Key: R7UUOLFDI2ZI9PRCQ53K

7.2.1.2. Request Headers

NameDescriptionTypeRequired

X-Auth-User

The key Ceph Object Gateway username to authenticate.

String

Yes

X-Auth-Key

The key associated to a Ceph Object Gateway username.

String

Yes

7.2.1.3. Response Headers

The response from the server should include an X-Auth-Token value. The response may also contain a X-Storage-Url that provides the {api version}/{account} prefix that is specified in other requests throughout the API documentation.

NameDescriptionType

X-Storage-Token

The authorization token for the X-Auth-User specified in the request.

String

X-Storage-Url

The URL and {api version}/{account} path for the user.

String

A typical response looks like this:

HTTP/1.1 204 No Content
Date: Mon, 16 Jul 2012 11:05:33 GMT
Server: swift
X-Storage-Url: https://swift.radosgwhost.com/v1/ACCT-12345
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
Content-Length: 0
Content-Type: text/plain; charset=UTF-8

7.3. Service Operations

To retrieve data about our Swift-compatible service, you may execute GET requests using the X-Storage-Url value retrieved during authentication.

7.3.1. List Containers

A GET request that specifies the API version and the account will return a list of containers for a particular user account. Since the request returns a particular user’s containers, the request requires an authentication token. The request cannot be made anonymously.

7.3.1.1. Syntax

GET /{api version}/{account} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.3.1.2. Request Parameters

NameDescriptionTypeRequiredValid Values

limit

Limits the number of results to the specified value.

Integer

No

N/A

format

Defines the format of the result.

String

No

json or xml

marker

Returns a list of results greater than the marker value.

String

No

N/A

7.3.1.3. Response Entities

The response contains a list of containers, or returns with an HTTP 204 response code

NameDescriptionType

account

A list for account information.

Container

container

The list of containers.

Container

name

The name of a container.

String

bytes

The size of the container.

Integer

7.4. Container Operations

A container is a mechanism for storing data objects. An account may have many containers, but container names must be unique. This API enables a client to create a container, set access controls and metadata, retrieve a container’s contents, and delete a container. Since this API makes requests related to information in a particular user’s account, all requests in this API must be authenticated unless a container’s access control is deliberately made publicly accessible (i.e., allows anonymous requests).

Note

The Amazon S3 API uses the term 'bucket' to describe a data container. When you hear someone refer to a 'bucket' within the Swift API, the term 'bucket' may be construed as the equivalent of the term 'container.'

One facet of object storage is that it does not support hierarchical paths or directories. Instead, it supports one level consisting of one or more containers, where each container may have objects. The RADOS Gateway’s Swift-compatible API supports the notion of 'pseudo-hierarchical containers,' which is a means of using object naming to emulate a container (or directory) hierarchy without actually implementing one in the storage system. You may name objects with pseudo-hierarchical names (e.g., photos/buildings/empire-state.jpg), but container names cannot contain a forward slash (/) character.

7.4.1. Create a Container

To create a new container, make a PUT request with the API version, account, and the name of the new container. The container name must be unique, must not contain a forward-slash (/) character, and should be less than 256 bytes. You may include access control headers and metadata headers in the request. You may also include a storage policy identifying a key for a set of placement pools (e.g., execute radosgw-admin zone get to see a list of available keys under placement_pools). A storage policy enables you to specify a special set of pools for the container (e.g., SSD-based storage). The operation is idempotent; that is, if you make a request to create a container that already exists, it will return with a HTTP 202 return code, but will not create another container.

7.4.1.1. Syntax

PUT /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}
X-Container-Read: {comma-separated-uids}
X-Container-Write: {comma-separated-uids}
X-Container-Meta-{key}: {value}
X-Storage-Policy: {placement-pools-key}

7.4.1.2. Headers

NameDescriptionTypeRequired

X-Container-Read

The user IDs with read permissions for the container.

Comma-separated string values of user IDs.

No

X-Container-Write

The user IDs with write permissions for the container.

Comma-separated string values of user IDs.

No

X-Container-Meta-{key}

A user-defined meta data key that takes an arbitrary string value.

String

No

X-Storage-Policy

The key that identifies the storage policy under placement_pools for the Ceph Object Gateway. Execute radosgw-admin zone get for available keys.

String

No

7.4.1.3. HTTP Response

If a container with the same name already exists, and the user is the container owner then the operation will succeed. Otherwise the operation will fail.

NameDescriptionStatus Code

409

The container already exists under a different user’s ownership.

BucketAlreadyExists

7.4.2. List a Container’s Objects

To list the objects within a container, make a GET request with the with the API version, account, and the name of the container. You can specify query parameters to filter the full list, or leave out the parameters to return a list of the first 10,000 object names stored in the container.

7.4.2.1. Syntax

GET /{api version}/{container} HTTP/1.1
 Host: {fqdn}
 X-Auth-Token: {auth-token}

7.4.2.2. Parameters

NameDescriptionTypeValid ValuesRequired

format

Defines the format of the result.

String

json or xml

No

prefix

Limits the result set to objects beginning with the specified prefix.

String

N/A

No

marker

Returns a list of results greater than the marker value.

String

N/A

No

limit

Limits the number of results to the specified value.

Integer

0 - 10,000

No

delimiter

The delimiter between the prefix and the rest of the object name.

String

N/A

No

path

The pseudo-hierarchical path of the objects.

String

N/A

No

7.4.2.3. Response Entities

NameDescriptionType

container

The container.

Container

object

An object within the container.

Container

name

The name of an object within the container.

String

hash

A hash code of the object’s contents.

String

last_modified

The last time the object’s contents were modified.

Date

content_type

The type of content within the object.

String

7.4.3. Update a Container’s ACLs

When a user creates a container, the user has read and write access to the container by default. To allow other users to read a container’s contents or write to a container, you must specifically enable the user. You may also specify * in the X-Container-Read or X-Container-Write settings, which effectively enables all users to either read from or write to the container. Setting * makes the container public. That is it enables anonymous users to either read from or write to the container.

7.4.3.1. Syntax

POST /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
 X-Auth-Token: {auth-token}
 X-Container-Read: *
 X-Container-Write: {uid1}, {uid2}, {uid3}

7.4.3.2. Request Headers

NameDescriptionTypeRequired

X-Container-Read

The user IDs with read permissions for the container.

Comma-separated string values of user IDs.

No

X-Container-Write

The user IDs with write permissions for the container.

Comma-separated string values of user IDs.

No

7.4.4. Add/Update Container Metadata

To add metadata to a container, make a POST request with the API version, account, and container name. You must have write permissions on the container to add or update metadata.

7.4.4.1. Syntax

POST /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
 X-Auth-Token: {auth-token}
 X-Container-Meta-Color: red
 X-Container-Meta-Taste: salty

7.4.4.2. Request Headers

NameDescriptionTypeRequired

X-Container-Meta-{key}

A user-defined meta data key that takes an arbitrary string value.

String

No

7.4.5. Delete a Container

To delete a container, make a DELETE request with the API version, account, and the name of the container. The container must be empty. If you’d like to check if the container is empty, execute a HEAD request against the container. Once you’ve successfully removed the container, you’ll be able to reuse the container name.

7.4.5.1. Syntax

DELETE /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.4.5.2. HTTP Response

NameDescriptionStatus Code

204

The container was removed.

NoContent

7.5. Object Operations

An object is a container for storing data and metadata. A container may have many objects, but the object names must be unique. This API enables a client to create an object, set access controls and metadata, retrieve an object’s data and metadata, and delete an object. Since this API makes requests related to information in a particular user’s account, all requests in this API must be authenticated unless the container or object’s access control is deliberately made publicly accessible (i.e., allows anonymous requests).

7.5.1. Create/Update an Object

To create a new object, make a PUT request with the API version, account, container name and the name of the new object. You must have write permission on the container to create or update an object. The object name must be unique within the container. The PUT request is not idempotent, so if you do not use a unique name, the request will update the object. However, you may use pseudo-hierarchical syntax in your object name to distinguish it from another object of the same name if it is under a different pseudo-hierarchical directory. You may include access control headers and metadata headers in the request.

7.5.1.1. Syntax

PUT /{api version}/{account}/{container}/{object} HTTP/1.1
 Host: {fqdn}
 X-Auth-Token: {auth-token}

7.5.1.2. Request Headers

NameDescriptionTypeRequiredValid Values

ETag

An MD5 hash of the object’s contents. Recommended.

String

No

N/A

Content-Type

The type of content the object contains.

String

No

N/A

Transfer-Encoding

Indicates whether the object is part of a larger aggregate object.

String

No

chunked

7.5.2. Copy an Object

Copying an object allows you to make a server-side copy of an object, so that you don’t have to download it and upload it under another container/name. To copy the contents of one object to another object, you may make either a PUT request or a COPY request with the API version, account, and the container name. For a PUT request, use the destination container and object name in the request, and the source container and object in the request header. For a Copy request, use the source container and object in the request, and the destination container and object in the request header. You must have write permission on the container to copy an object. The destination object name must be unique within the container. The request is not idempotent, so if you do not use a unique name, the request will update the destination object. However, you may use pseudo-hierarchical syntax in your object name to distinguish the destination object from the source object of the same name if it is under a different pseudo-hierarchical directory. You may include access control headers and metadata headers in the request.

7.5.2.1. Syntax

PUT /{api version}/{account}/{dest-container}/{dest-object} HTTP/1.1
X-Copy-From: {source-container}/{source-object}
Host: {fqdn}
X-Auth-Token: {auth-token}

or alternatively:

COPY /{api version}/{account}/{source-container}/{source-object} HTTP/1.1
Destination: {dest-container}/{dest-object}

7.5.2.2. Request Headers

NameDescriptionTypeRequired

X-Copy-From

Used with a PUT request to define the source container/object path.

String

Yes, if using PUT

Destination

Used with a COPY request to define the destination container/object path.

String

Yes, if using COPY

If-Modified-Since

Only copies if modified since the date/time of the source object’s last_modified attribute.

Date

No

If-Unmodified-Since

Only copies if not modified since the date/time of the source object’s last_modified attribute.

Date

No

Copy-If-Match

Copies only if the ETag in the request matches the source object’s ETag.

ETag.

No

Copy-If-None-Match

Copies only if the ETag in the request does not match the source object’s ETag.

ETag.

No

7.5.3. Delete an Object

To delete an object, make a DELETE request with the API version, account, container and object name. You must have write permissions on the container to delete an object within it. Once you’ve successfully deleted the object, you’ll be able to reuse the object name.

7.5.3.1. Syntax

DELETE /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.5.4. Get an Object

To retrieve an object, make a GET request with the API version, account, container and object name. You must have read permissions on the container to retrieve an object within it.

7.5.4.1. Syntax

GET /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.5.4.2. Request Headers

NameDescriptionTypeRequired

range

To retrieve a subset of an object’s contents, you may specify a byte range.

Date

No

If-Modified-Since

Only copies if modified since the date/time of the source object’s last_modified attribute.

Date

No

If-Unmodified-Since

Only copies if not modified since the date/time of the source object’s last_modified attribute.

Date

No

Copy-If-Match

Copies only if the ETag in the request matches the source object’s ETag.

ETag.

No

Copy-If-None-Match

Copies only if the ETag in the request does not match the source object’s ETag.

ETag.

No

7.5.4.3. Response Headers

NameDescription

Content-Range

The range of the subset of object contents. Returned only if the range header field was specified in the request.

7.5.5. Get Object Metadata

To retrieve an object’s metadata, make a HEAD request with the API version, account, container and object name. You must have read permissions on the container to retrieve metadata from an object within the container. This request returns the same header information as the request for the object itself, but it does not return the object’s data.

7.5.5.1. Syntax

HEAD /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.5.6. Add/Update Object Metadata

To add metadata to an object, make a POST request with the API version, account, container and object name. You must have write permissions on the parent container to add or update metadata.

7.5.6.1. Syntax

POST /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.5.6.2. Request Headers

NameDescriptionTypeRequired

X-Object-Meta-{key}

A user-defined meta data key that takes an arbitrary string value.

String

No

7.6. Temp URL Operations

To allow temporary access (for eg for GET requests) to objects without the need to share credentials, temp url functionality is supported by swift endpoint of radosgw. For this functionality, initially the value of X-Account-Meta-Temp-URL-Key and optionally X-Account-Meta-Temp-URL-Key-2 should be set. The Temp URL functionality relies on a HMAC-SHA1 signature against these secret keys.

7.6.1. POST Temp-URL Keys

A POST request to the swift account with the required Key will set the secret temp url key for the account against which temporary url access can be provided to accounts. Up to two keys are supported, and signatures are checked against both the keys, if present, so that keys can be rotated without invalidating the temporary urls.

7.6.1.1. Syntax

POST /{api version}/{account} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

7.6.1.2. Request Headers

NameDescriptionTypeRequired

X-Account-Meta-Temp-URL-Key

A user-defined key that takes an arbitrary string value.

String

Yes

X-Account-Meta-Temp-URL-Key-2

A user-defined key that takes an arbitrary string value.

String

No

7.6.2. GET Temp-URL Objects

Temporary URL uses a cryptographic HMAC-SHA1 signature, which includes the following elements:

  1. The value of the Request method, "GET" for instance
  2. The expiry time, in format of seconds since the epoch, ie Unix time
  3. The request path starting from "v1" onwards

The above items are normalized with newlines appended between them, and a HMAC is generated using the SHA-1 hashing algorithm against one of the Temp URL Keys posted earlier.

A sample python script to demonstrate the above is given below:

import hmac
from hashlib import sha1
from time import time

method = 'GET'
host = 'https://objectstore.example.com'
duration_in_seconds = 300  # Duration for which the url is valid
expires = int(time() + duration_in_seconds)
path = '/v1/your-bucket/your-object'
key = 'secret'
hmac_body = '%s\n%s\n%s' % (method, expires, path)
hmac_body = hmac.new(key, hmac_body, sha1).hexdigest()
sig = hmac.new(key, hmac_body, sha1).hexdigest()
rest_uri = "{host}{path}?temp_url_sig={sig}&temp_url_expires={expires}".format(
     host=host, path=path, sig=sig, expires=expires)
print rest_uri

# Example Output
# https://objectstore.example.com/v1/your-bucket/your-object?temp_url_sig=ff4657876227fc6025f04fcf1e82818266d022c6&temp_url_expires=1423200992