Red Hat Training

A Red Hat training course is available for Red Hat Ceph Storage

Chapter 6. Object Gateway S3 API

Red Hat Ceph Object Gateway supports a RESTful API that is compatible with the basic data access model of the Amazon S3 API.

6.1. API

6.1.1. Feature Support

The following table describes the support status for current Amazon S3 functional features:

Feature	Status	Remarks
List Buckets	Supported
Create Bucket	Supported	Different set of canned ACLs
Get Bucket	Supported
Get Bucket Location	Supported
Delete Bucket	Supported
Bucket ACLs (Get, Put)	Supported	Different set of canned ACLs
Bucket Object Versions	Supported
Get Bucket Info (HEAD)	Supported
List Bucket Multipart Uploads	Supported
Bucket Lifecycle	Not Supported
Policy (Buckets, Objects)	Not Supported	ACLs are supported
Bucket Website	Not Supported
Bucket Notification	Not Supported
Bucket Request Payment	Not Supported
Put Object	Supported
Delete Object	Supported
Get Object	Supported
Object ACLs (Get, Put)	Supported
Get Object Info (HEAD)	Supported
Copy Object	Supported
Initiate Multipart Upload	Supported
Initiage Multipart Upload Part	Supported
List Multipart Upload Parts	Supported
Complete Multipart Upload	Supported
Abort Multipart Upload	Supported
Multipart Uploads	Supported	(missing Copy Part)

6.1.2. Unsupported Header Fields

The following common request header fields are not supported:

Name	Type
x-amz-security-token	Request
Server	Response
x-amz-delete-marker	Response
x-amz-id-2	Response
x-amz-request-id	Response
x-amz-version-id	Response

6.2. Common

6.2.1. Bucket and Host Name

There are two different modes of accessing the buckets. The first (preferred) method identifies the bucket as the top-level directory in the URI.

GET /mybucket HTTP/1.1
Host: cname.domain.com

The second method identifies the bucket via a virtual bucket host name. For example:

GET / HTTP/1.1
Host: mybucket.cname.domain.com

Tip

We prefer the first method, because the second method requires expensive domain certification and DNS wild cards.

6.2.2. Common Request Headers

Request Header	Description
`CONTENT_LENGTH`	Length of the request body.
`DATE`	Request time and date (in UTC).
`HOST`	The name of the host server.
`AUTHORIZATION`	Authorization token.

6.2.3. Common Response Status

HTTP Status	Response Code
`100`	Continue
`200`	Success
`201`	Created
`202`	Accepted
`204`	NoContent
`206`	Partial content
`304`	NotModified
`400`	InvalidArgument
`400`	InvalidDigest
`400`	BadDigest
`400`	InvalidBucketName
`400`	InvalidObjectName
`400`	UnresolvableGrantByEmailAddress
`400`	InvalidPart
`400`	InvalidPartOrder
`400`	RequestTimeout
`400`	EntityTooLarge
`403`	AccessDenied
`403`	UserSuspended
`403`	RequestTimeTooSkewed
`404`	NoSuchKey
`404`	NoSuchBucket
`404`	NoSuchUpload
`405`	MethodNotAllowed
`408`	RequestTimeout
`409`	BucketAlreadyExists
`409`	BucketNotEmpty
`411`	MissingContentLength
`412`	PreconditionFailed
`416`	InvalidRange
`422`	UnprocessableEntity
`500`	InternalError

6.3. Authentication and ACLs

Requests to the Ceph Object Gateway can be either authenticated or unauthenticated. Ceph Object Gateway assumes unauthenticated requests are sent by an anonymous user. Ceph Object Gateway supports canned ACLs.

6.3.1. Authentication

For most use cases, clients use existing open source libraries like the Amazon SDK’s AmazonS3Client for Java, Python Boto, etc. where you simply pass in the access key and secret key, and the library builds the request header and authentication signature for you. However, you can create your own requests and sign them too.

Authenticating a request requires including an access key and a base 64-encoded Hash-based Message Authentication Code (HMAC) in the request before it is sent to the Ceph Object Gateway server. Ceph Object Gateway uses an S3-compatible authentication approach.

HTTP/1.1
PUT /buckets/bucket/object.mpeg
Host: cname.domain.com
Date: Mon, 2 Jan 2012 00:01:01 +0000
Content-Encoding: mpeg
Content-Length: 9999999

Authorization: AWS {access-key}:{hash-of-header-and-secret}

In the foregoing example, replace {access-key} with the value for your access key ID followed by a colon (:). Replace {hash-of-header-and-secret} with a hash of a canonicalized header string and the secret corresponding to the access key ID.

To generate the hash of the header string and secret, you must:

Get the value of the header string.
Normalize the request header string into canonical form.
Generate an HMAC using a SHA-1 hashing algorithm.
Encode the hmac result as base-64.

To normalize the header into canonical form:

Get all content- headers.
Remove all content- headers except for content-type and content-md5.
Ensure the content- header names are lowercase.
Sort the content- headers lexicographically.
Ensure you have a Date header AND ensure the specified date uses GMT and not an offset.
Get all headers beginning with x-amz-.
Ensure that the x-amz- headers are all lowercase.
Sort the x-amz- headers lexicographically.
Combine multiple instances of the same field name into a single field and separate the field values with a comma.
Replace white space and line breaks in header values with a single space.
Remove white space before and after colons.
Append a new line after each header.
Merge the headers back into the request header.

Replace the {hash-of-header-and-secret} with the base-64 encoded HMAC string.

For additional details, consult the Signing and Authenticating REST Requests section of Amazon Simple Storage Service documentation.

6.3.2. Access Control Lists (ACLs)

Ceph Object Gateway supports S3-compatible ACL functionality. An ACL is a list of access grants that specify which operations a user can perform on a bucket or on an object. Each grant has a different meaning when applied to a bucket versus applied to an object:

Permission	Bucket	Object
`READ`	Grantee can list the objects in the bucket.	Grantee can read the object.
`WRITE`	Grantee can write or delete objects in the bucket.	N/A
`READ_ACP`	Grantee can read bucket ACL.	Grantee can read the object ACL.
`WRITE_ACP`	Grantee can write bucket ACL.	Grantee can write to the object ACL.
`FULL_CONTROL`	Grantee has full permissions for object in the bucket.	Grantee can read or write to the object ACL.

6.4. Service Operations

6.4.1. List Buckets

GET / returns a list of buckets created by the user making the request. GET / only returns buckets created by an authenticated user. You cannot make an anonymous request.

6.4.1.1. Syntax

GET / HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.4.1.2. Response Entities

Name	Type	Description
`Buckets`	Container	Container for list of buckets.
`Bucket`	Container	Container for bucket information.
`Name`	String	Bucket name.
`CreationDate`	Date	UTC time when the bucket was created.
`ListAllMyBucketsResult`	Container	A container for the result.
`Owner`	Container	A container for the bucket owner’s `ID` and `DisplayName`.
`ID`	String	The bucket owner’s ID.
`DisplayName`	String	The bucket owner’s display name.

6.5. Bucket Operations

6.5.1. PUT Bucket

Creates a new bucket. To create a bucket, you must have a user ID and a valid AWS Access Key ID to authenticate requests. You may not create buckets as an anonymous user.

6.5.1.1. Constraints

In general, bucket names should follow domain name constraints.

Bucket names must be unique.
Bucket names must begin and end with a lowercase letter.
Bucket names may contain a dash (-).

6.5.1.2. Syntax

PUT /{bucket} HTTP/1.1
Host: cname.domain.com
x-amz-acl: public-read-write

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.5.1.3. Parameters

Name	Description	Valid Values	Required
`x-amz-acl`	Canned ACLs.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No

6.5.1.4. HTTP Response

If the bucket name is unique, within constraints and unused, the operation will succeed. If a bucket with the same name already exists and the user is the bucket owner, the operation will succeed. If the bucket name is already in use, the operation will fail.

HTTP Status	Status Code	Description
`409`	BucketAlreadyExists	Bucket already exists under different user’s ownership.

6.5.2. DELETE Bucket

Deletes a bucket. You can reuse bucket names following a successful bucket removal.

6.5.2.1. Syntax

DELETE /{bucket} HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.5.2.2. HTTP Response

HTTP Status	Status Code	Description
`204`	No Content	Bucket removed.

6.5.3. GET Bucket

Returns a list of bucket objects.

6.5.3.1. Syntax

GET /{bucket}?max-keys=25 HTTP/1.1
Host: cname.domain.com

6.5.3.2. Parameters

Name	Type	Description
`prefix`	String	Only returns objects that contain the specified prefix.
`delimiter`	String	The delimiter between the prefix and the rest of the object name.
`marker`	String	A beginning index for the list of objects returned.
`max-keys`	Integer	The maximum number of keys to return. Default is 1000.

6.5.3.3. HTTP Response

HTTP Status	Status Code	Description
`200`	OK	Buckets retrieved

6.5.3.4. Bucket Response Entities

GET /{bucket} returns a container for buckets with the following fields.

Name	Type	Description
`ListBucketResult`	Entity	The container for the list of objects.
`Name`	String	The name of the bucket whose contents will be returned.
`Prefix`	String	A prefix for the object keys.
`Marker`	String	A beginning index for the list of objects returned.
`MaxKeys`	Integer	The maximum number of keys returned.
`Delimiter`	String	If set, objects with the same prefix will appear in the `CommonPrefixes` list.
`IsTruncated`	Boolean	If `true`, only a subset of the bucket’s contents were returned.
`CommonPrefixes`	Container	If multiple objects contain the same prefix, they will appear in this list.

6.5.3.5. Object Response Entities

The ListBucketResult contains objects, where each object is within a Contents container.

Name	Type	Description
`Contents`	Object	A container for the object.
`Key`	String	The object’s key.
`LastModified`	Date	The object’s last-modified date/time.
`ETag`	String	An MD-5 hash of the object. (entity tag)
`Size`	Integer	The object’s size.
`StorageClass`	String	Should always return `STANDARD`.

6.5.4. Get Bucket Location

Retrieves the bucket’s region. The user needs to be the bucket owner to call this. A bucket can be constrained to a region by providing LocationConstraint during a PUT request.

6.5.4.1. Syntax

Add the location subresource to bucket resource as shown below.

GET /{bucket}?location HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.5.4.2. Response Entities

Name	Type	Description
`LocationConstraint`	String	The region where bucket resides, empty string for defult region

6.5.5. Get Bucket ACLs

Retrieves the bucket access control list. The user needs to be the bucket owner or to have been granted READ_ACP permission on the bucket.

6.5.5.1. Syntax

Add the acl subresource to the bucket request as shown below.

GET /{bucket}?acl HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.5.5.2. Response Entities

Name	Type	Description
`AccessControlPolicy`	Container	A container for the response.
`AccessControlList`	Container	A container for the ACL information.
`Owner`	Container	A container for the bucket owner’s `ID` and `DisplayName`.
`ID`	String	The bucket owner’s ID.
`DisplayName`	String	The bucket owner’s display name.
`Grant`	Container	A container for `Grantee` and `Permission`.
`Grantee`	Container	A container for the `DisplayName` and `ID` of the user receiving a grant of permission.
`Permission`	String	The permission given to the `Grantee` bucket.

6.5.6. PUT Bucket ACLs

Sets an access control to an existing bucket. The user needs to be the bucket owner or to have been granted WRITE_ACP permission on the bucket.

6.5.6.1. Syntax

Add the acl subresource to the bucket request as shown below.

PUT /{bucket}?acl HTTP/1.1

6.5.6.2. Request Entities

Name	Type	Description
`AccessControlPolicy`	Container	A container for the request.
`AccessControlList`	Container	A container for the ACL information.
`Owner`	Container	A container for the bucket owner’s `ID` and `DisplayName`.
`ID`	String	The bucket owner’s ID.
`DisplayName`	String	The bucket owner’s display name.
`Grant`	Container	A container for `Grantee` and `Permission`.
`Grantee`	Container	A container for the `DisplayName` and `ID` of the user receiving a grant of permission.
`Permission`	String	The permission given to the `Grantee` bucket.

6.5.7. List Bucket Object Versions

Returns a list of metadata about all the version of objects within a bucket. Requires READ access to the bucket.

6.5.7.1. Syntax

Add the versions subresource to the bucket request as shown below.

GET /{bucket}?versions HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.5.7.2. Parameters

You may specify parameters for GET /{bucket}?versions, but none of them are required.

Name	Type	Description
`prefix`	String	Returns in-progress uploads whose keys contains the specified prefix.
`delimiter`	String	The delimiter between the prefix and the rest of the object name.
`key-marker`	String	The beginning marker for the list of uploads.
`max-keys`	Integer	The maximum number of in-progress uploads. The default is 1000.
`version-id-marker`	String	Specifies the object version to begin the list.

6.5.7.3. Response Entities

Name	Type	Description
`KeyMarker`	String	The key marker specified by the `key-marker` request parameter (if any).
`NextKeyMarker`	String	The key marker to use in a subsequent request if `IsTruncated` is `true`.
`NextUploadIdMarker`	String	The upload ID marker to use in a subsequent request if `IsTruncated` is `true`.
`IsTruncated`	Boolean	If `true`, only a subset of the bucket’s upload contents were returned.
`Size`	Integer	The size of the uploaded part.
`DisplayName`	String	The owners’s display name.
`ID`	String	The owners’s ID.
`Owner`	Container	A container for the `ID` and `DisplayName` of the user who owns the object.
`StorageClass`	String	The method used to store the resulting object. `STANDARD` or `REDUCED_REDUNDANCY`
`Version`	Container	Container for the version information.
`versionId`	String	Version ID of an object.
`versionIdMarker`	String	The last version of the key in a truncated response.

6.5.8. List Bucket Multipart Uploads

GET /?uploads returns a list of the current in-progress multipart uploads—i.e., the application initiates a multipart upload, but the service hasn’t completed all the uploads yet.

6.5.8.1. Syntax

GET /{bucket}?uploads HTTP/1.1

6.5.8.2. Parameters

You may specify parameters for GET /{bucket}?uploads, but none of them are required.

Name	Type	Description
`prefix`	String	Returns in-progress uploads whose keys contains the specified prefix.
`delimiter`	String	The delimiter between the prefix and the rest of the object name.
`key-marker`	String	The beginning marker for the list of uploads.
`max-keys`	Integer	The maximum number of in-progress uploads. The default is 1000.
`max-uploads`	Integer	The maximum number of multipart uploads. The range from 1-1000. The default is 1000.
`version-id-marker`	String	Ignored if `key-marker` isn’t specified. Specifies the `ID` of first upload to list in lexicographical order at or following the `ID`.

6.5.8.3. Response Entities

Name	Type	Description
`ListMultipartUploadsResult`	Container	A container for the results.
`ListMultipartUploadsResult.Prefix`	String	The prefix specified by the `prefix` request parameter (if any).
`Bucket`	String	The bucket that will receive the bucket contents.
`KeyMarker`	String	The key marker specified by the `key-marker` request parameter (if any).
`UploadIdMarker`	String	The marker specified by the `upload-id-marker` request parameter (if any).
`NextKeyMarker`	String	The key marker to use in a subsequent request if `IsTruncated` is `true`.
`NextUploadIdMarker`	String	The upload ID marker to use in a subsequent request if `IsTruncated` is `true`.
`MaxUploads`	Integer	The max uploads specified by the `max-uploads` request parameter.
`Delimiter`	String	If set, objects with the same prefix will appear in the `CommonPrefixes` list.
`IsTruncated`	Boolean	If `true`, only a subset of the bucket’s upload contents were returned.
`Upload`	Container	A container for `Key`, `UploadId`, `InitiatorOwner`, `StorageClass`, and `Initiated` elements.
`Key`	String	The key of the object once the multipart upload is complete.
`UploadId`	String	The `ID` that identifies the multipart upload.
`Initiator`	Container	Contains the `ID` and `DisplayName` of the user who initiated the upload.
`DisplayName`	String	The initiator’s display name.
`ID`	String	The initiator’s ID.
`Owner`	Container	A container for the `ID` and `DisplayName` of the user who owns the uploaded object.
`StorageClass`	String	The method used to store the resulting object. `STANDARD` or `REDUCED_REDUNDANCY`
`Initiated`	Date	The date and time the user initiated the upload.
`CommonPrefixes`	Container	If multiple objects contain the same prefix, they will appear in this list.
`CommonPrefixes.Prefix`	String	The substring of the key after the prefix as defined by the `prefix` request parameter.

6.6. Object Operations

6.6.1. PUT Object

Adds an object to a bucket. You must have write permissions on the bucket to perform this operation.

6.6.1.1. Syntax

PUT /{bucket}/{object} HTTP/1.1

6.6.1.2. Request Headers

Name	Description	Valid Values	Required
content-md5	A base64 encoded MD-5 hash of the message.	A string. No defaults or constraints.	No
content-type	A standard MIME type.	Any MIME type. Default: `binary/octet-stream`	No
x-amz-meta-<…>	User metadata. Stored with the object.	A string up to 8kb. No defaults.	No
x-amz-acl	A canned ACL.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No

6.6.1.3. Response Headers

Name	Description
x-amz-version-id	Returns the version ID or null.

6.6.2. Copy Object

To copy an object, use PUT and specify a destination bucket and the object name.

6.6.2.1. Syntax

PUT /{dest-bucket}/{dest-object} HTTP/1.1
x-amz-copy-source: {source-bucket}/{source-object}

6.6.2.2. Request Headers

Name	Description	Valid Values	Required
x-amz-copy-source	The source bucket name + object name.	{bucket}/{obj}	Yes
x-amz-acl	A canned ACL.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No
x-amz-copy-if-modified-since	Copies only if modified since the timestamp.	Timestamp	No
x-amz-copy-if-unmodified-since	Copies only if unmodified since the timestamp.	Timestamp	No
x-amz-copy-if-match	Copies only if object ETag matches ETag.	Entity Tag	No
x-amz-copy-if-none-match	Copies only if object ETag doesn’t match.	Entity Tag	No

6.6.2.3. Response Entities

Name	Type	Description
CopyObjectResult	Container	A container for the response elements.
LastModified	Date	The last modified date of the source object.
Etag	String	The ETag of the new object.

6.6.3. Remove Object

Removes an object. Requires WRITE permission set on the containing bucket.

6.6.3.1. Syntax

Deletes an object. If object versioning is on, it creates a marker.

DELETE /{bucket}/{object} HTTP/1.1

To delete an object when versioning is on, you must specify the versionId subresource and the version of the object to delete.

DELETE /{bucket}/{object}?versionId={versionID} HTTP/1.1

6.6.4. Get Object

Retrieves an object from a bucket.

6.6.4.1. Syntax

GET /{bucket}/{object} HTTP/1.1

Add the versionId subresource to retrieve a particular version of the object.

GET /{bucket}/{object}?versionId={versionID} HTTP/1.1

6.6.4.2. Request Headers

Name	Description	Valid Values	Required
range	The range of the object to retrieve.	Range: bytes=beginbyte-endbyte	No
if-modified-since	Gets only if modified since the timestamp.	Timestamp	No
if-unmodified-since	Gets only if not modified since the timestamp.	Timestamp	No
if-match	Gets only if object ETag matches ETag.	Entity Tag	No
if-none-match	Gets only if object ETag matches ETag.	Entity Tag	No

6.6.4.3. Response Headers

Name	Description
Content-Range	Data range, will only be returned if the range header field was specified in the request
x-amz-version-id	Returns the version ID or null.

6.6.5. Get Object Information

Returns information about an object. This request will return the same header information as with the Get Object request, but will include the metadata only, not the object data payload.

6.6.5.1. Syntax

Retrieves the current version of the object.

HEAD /{bucket}/{object} HTTP/1.1

Add the versionId subresource to retrieve info for a particular version.

HEAD /{bucket}/{object}?versionId={versionID} HTTP/1.1

6.6.5.2. Request Headers

Name	Description	Valid Values	Required
range	The range of the object to retrieve.	Range: bytes=beginbyte-endbyte	No
if-modified-since	Gets only if modified since the timestamp.	Timestamp	No
if-unmodified-since	Gets only if not modified since the timestamp.	Timestamp	No
if-match	Gets only if object ETag matches ETag.	Entity Tag	No
if-none-match	Gets only if object ETag matches ETag.	Entity Tag	No

6.6.5.3. Response Headers

Name	Description
x-amz-version-id	Returns the version ID or null.

6.6.6. Get Object ACL

6.6.6.1. Syntax

Returns the ACL for the current version of the object.

GET /{bucket}/{object}?acl HTTP/1.1

Add the versionId subresource to retrieve the ACL for a particular version.

GET /{bucket}/{object}versionId={versionID}&acl HTTP/1.1

6.6.6.2. Response Headers

Name	Description
x-amz-version-id	Returns the version ID or null.

6.6.6.3. Response Entities

Name	Type	Description
`AccessControlPolicy`	Container	A container for the response.
`AccessControlList`	Container	A container for the ACL information.
`Owner`	Container	A container for the object owner’s `ID` and `DisplayName`.
`ID`	String	The object owner’s ID.
`DisplayName`	String	The object owner’s display name.
`Grant`	Container	A container for `Grantee` and `Permission`.
`Grantee`	Container	A container for the `DisplayName` and `ID` of the user receiving a grant of permission.
`Permission`	String	The permission given to the `Grantee` object.

6.6.7. Set Object ACL

Sets an object ACL for the current version of the object.

6.6.7.1. Syntax

PUT /{bucket}/{object}?acl

6.6.7.2. Request Entities

Name	Type	Description
`AccessControlPolicy`	Container	A container for the response.
`AccessControlList`	Container	A container for the ACL information.
`Owner`	Container	A container for the object owner’s `ID` and `DisplayName`.
`ID`	String	The object owner’s ID.
`DisplayName`	String	The object owner’s display name.
`Grant`	Container	A container for `Grantee` and `Permission`.
`Grantee`	Container	A container for the `DisplayName` and `ID` of the user receiving a grant of permission.
`Permission`	String	The permission given to the `Grantee` object.

6.6.8. Initiate Multipart Upload

Initiates a multi-part upload process. Returns a UploadId, which you may specify when adding additional parts, listing parts, and completing or abandoning a multi-part upload.

6.6.8.1. Syntax

POST /{bucket}/{object}?uploads

6.6.8.2. Request Headers

Name	Description	Valid Values	Required
`content-md5`	A base64 encoded MD-5 hash of the message.	A string. No defaults or constraints.	No
`content-type`	A standard MIME type.	Any MIME type. Default: `binary/octet-stream`	No
`x-amz-meta-<…>`	User metadata. Stored with the object.	A string up to 8kb. No defaults.	No
`x-amz-acl`	A canned ACL.	`private`, `public-read`, `public-read-write`, `authenticated-read`	No

6.6.8.3. Response Entities

Name	Type	Description
`InitiatedMultipartUploadsResult`	Container	A container for the results.
`Bucket`	String	The bucket that will receive the object contents.
`Key`	String	The key specified by the `key` request parameter (if any).
`UploadId`	String	The ID specified by the `upload-id` request parameter identifying the multipart upload (if any).

6.6.9. Multipart Upload Part

Adds a part to a multi-part upload.

6.6.9.1. Syntax

Specify the uploadId subresource and the upload ID to add a part to a multi-part upload.

PUT /{bucket}/{object}?partNumber=&uploadId={upload-id} HTTP/1.1

6.6.9.2. HTTP Response

The following HTTP response may be returned:

HTTP Status	Status Code	Description
`404`	NoSuchUpload	Specified upload-id does not match any initiated upload on this object

6.6.10. List Multipart Upload Parts

6.6.10.1. Syntax

Specify the uploadId subresource and the upload ID to list the parts of a multi-part upload.

GET /{bucket}/{object}?uploadId={upload-id} HTTP/1.1

6.6.10.2. Response Entities

Name	Type	Description
`InitiatedMultipartUploadsResult`	Container	A container for the results.
`Bucket`	String	The bucket that will receive the object contents.
`Key`	String	The key specified by the `key` request parameter (if any).
`UploadId`	String	The ID specified by the `upload-id` request parameter identifying the multipart upload (if any).
`Initiator`	Container	Contains the `ID` and `DisplayName` of the user who initiated the upload.
`ID`	String	The initiator’s ID.
`DisplayName`	String	The initiator’s display name.
`Owner`	Container	A container for the `ID` and `DisplayName` of the user who owns the uploaded object.
`StorageClass`	String	The method used to store the resulting object. `STANDARD` or `REDUCED_REDUNDANCY`
`PartNumberMarker`	String	The part marker to use in a subsequent request if `IsTruncated` is `true`. Precedes the list.
`NextPartNumberMarker`	String	The next part marker to use in a subsequent request if `IsTruncated` is `true`. The end of the list.
`MaxParts`	Integer	The max parts allowed in the response as specified by the `max-parts` request parameter.
`IsTruncated`	Boolean	If `true`, only a subset of the object’s upload contents were returned.
`Part`	Container	A container for `Key`, `Part`, `InitiatorOwner`, `StorageClass`, and `Initiated` elements.
`PartNumber`	Integer	The identification number of the part.
`ETag`	String	The part’s entity tag.
`Size`	Integer	The size of the uploaded part.

6.6.11. Complete Multipart Upload

Assembles uploaded parts and creates a new object, thereby completing a multipart upload.

6.6.11.1. Syntax

Specify the uploadId subresource and the upload ID to complete a multi-part upload.

POST /{bucket}/{object}?uploadId= HTTP/1.1

6.6.11.2. Request Entities

Name	Type	Description	Required
`CompleteMultipartUpload`	Container	A container consisting of one or more parts.	Yes
`Part`	Container	A container for the `PartNumber` and `ETag`.	Yes
`PartNumber`	Integer	The identifier of the part.	Yes
`ETag`	String	The part’s entity tag.	Yes

6.6.11.3. Response Entities

Name	Type	Description
`CompleteMultipartUploadResult`	Container	A container for the response.
`Location`	URI	The resource identifier (path) of the new object.
`Bucket`	String	The name of the bucket that contains the new object.
`Key`	String	The object’s key.
`ETag`	String	The entity tag of the new object.

6.6.12. Abort Multipart Upload

Aborts a multipart upload.

6.6.12.1. Syntax

Specify the uploadId subresource and the upload ID to abort a multi-part upload.

DELETE /{bucket}/{object}?uploadId={upload-id} HTTP/1.1

6.7. Accessing the Gateway

You can use various programming languages to create a connection with the gateway server and do the bucket management tasks. There are different open source libraries available for these programming languages that are used for authentication with the gateway.

The sections mentioned below will describe the procedure for some of the popular programmimg languages.

6.7.1. Prerequisites

You have to follow some pre-requisites on your gateway node before attempting to access the gateway server. The pre-requisites are as follows:

Set up your gateway server properly by following the instructions mentioned in Install RHCS v1.3 Ceph Object Gateway for RHEL.
DO NOT modify the Ceph configuration file to use port 80 and let Civetweb use the default port 7480. Port 80 is required by Apache and it needs to be running and enabled.

Install and start Apache.

sudo yum install httpd -y
sudo systemctl start httpd
sudo systemctl enable httpd.service

Open port 80 on firewall.

sudo firewall-cmd --zone=public --add-port=80/tcp --permanent
sudo firewall-cmd --reload

Disable ssl.
If you have ssl enabled in your gateway node, please follow the steps mentioned in Disable SSL to disable it.
Add a wildcard to your DNS server that you are using for your gateway as mentioned in Add wildcard to DNS.
You can also set up your gateway node for local DNS caching. To do so, execute the following steps:
- Install and setup dnsmasq.
```
sudo yum install dnsmasq -y
echo "address=/.<FQDN_of_gateway_node>/<IP_of_gateway_node>" | sudo tee --append /etc/dnsmasq.conf
sudo systemctl start dnsmasq
sudo systemctl enable dnsmasq
```
  Replace <IP_of_gateway_node> and <FQDN_of_gateway_node> with the IP address and FQDN of your gateway node.
- Stop NetworkManager.
```
sudo systemctl stop NetworkManager
sudo systemctl disable NetworkManager
```
- Set your gateway server’s IP as the nameserver.
```
echo "DNS1=<IP_of_gateway_node>" | sudo tee --append /etc/sysconfig/network-scripts/ifcfg-eth0
echo "<IP_of_gateway_node> <FQDN_of_gateway_node>" | sudo tee --append /etc/hosts
sudo systemctl restart network
sudo systemctl enable network
sudo systemctl restart dnsmasq
```
  Replace <IP_of_gateway_node> and <FQDN_of_gateway_node> with the IP address and FQDN of your gateway node.
- Verify subdomain requests:
```
ping mybucket.<FQDN_of_gateway_node>
```
  Replace <FQDN_of_gateway_node> with the FQDN of your gateway node.
  Warning
  Setting up your gateway server for local DNS caching is for testing purposes only. You won’t be able to access outside network after doing this. It is strongly recommended to use a proper DNS server for your Ceph cluster and gateway node.
Create the radosgw user for S3 access carefully as mentioned in Create Radosgw user for S3 access and copy the generated access_key and secret_key. You will need these keys for S3 access and subsequent bucket management tasks.

6.7.2. Ruby AWS::S3 Examples (aws-s3 gem)

You can use Ruby programming language alongwith aws-s3 gem for S3 access. Execute the steps mentioned below in your gateway node for accessing the gateway with Ruby AWS::S3.

6.7.2.1. Setup Ruby

Execute the following steps to setup Ruby:

Install Ruby:
```
sudo yum install ruby -y
```
Note
The above command will install ruby and it’s essential dependencies like rubygems and ruby-libs too. If somehow the command doesn’t install all the dependencies, install them separately.
Install aws-s3:
```
sudo gem install aws-s3
```

6.7.2.2. Creating a connection

Create a project directory:
```
mkdir ruby_aws_s3
cd ruby_aws_s3
```
Create the connection file:
```
vim conn.rb
```

Paste the following contents in the conn.rb file:

#!/usr/bin/env ruby

require 'aws/s3'
require 'resolv-replace'

AWS::S3::Base.establish_connection!(
        :server            => '<FQDN_of_gateway_node>',
        :port           => '7480',
        :access_key_id     => 'my-access-key',
        :secret_access_key => 'my-secret-key'
)

Replace <FQDN_of_gateway_node> with the FQDN of you gateway node. Replace my-access-key and my-secret-key with the access_key and secret_key that was generated when you created the radosgw user for S3 access as mentioned in Create Radosgw user for S3 access.

An example connection file will look like the following:

#!/usr/bin/env ruby

require 'aws/s3'

require 'resolv-replace'

AWS::S3::Base.establish_connection!(
        :server            => 'testclient.englab.pnq.redhat.com',
        :port           => '7480',
        :access_key_id     => '98J4R9P22P5CDL65HKP8',
        :secret_access_key => '6C+jcaP0dp0+FZfrRNgyGA9EzRy25pURldwje049'
)

Save the file and exit the editor.

Make the file executable:
```
chmod +x conn.rb
```
Run the file:
```
./conn.rb | echo $?
```
If you have provided the values correctly in the file, the output of the command will be 0.

6.7.2.3. Creating a bucket

Create a new file:

vim create_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Bucket.create('my-new-bucket1')

Save the file and exit the editor.

Make the file executable:
```
chmod +x create_bucket.rb
```
Run the file:
```
./create_bucket.rb
```
If the output of the command is true it would mean that bucket my-new-bucket1 was created successfully.

6.7.2.4. Listing owned buckets

Create a new file:

vim list_owned_buckets.rb

Paste the following content into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Service.buckets.each do |bucket|
        puts "#{bucket.name}\t#{bucket.creation_date}"
end

Save the file and exit the editor.

Make the file executable:
```
chmod +x list_owned_buckets.rb
```

Run the file:

./list_owned_buckets.rb

The output should look something like this:

my-new-bucket1 2016-01-21 10:33:19 UTC

6.7.2.5. Creating an object

Create a new file:

vim create_object.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::S3Object.store(
        'hello.txt',
        'Hello World!',
        'my-new-bucket1',
        :content_type => 'text/plain'
)

Save the file and exit the editor.

Make the file executable:
```
chmod +x create_object.rb
```
Run the file:
```
./create_object.rb
```
This will create a file hello.txt with the string Hello World!.

6.7.2.6. Listing a Bucket’s Content

Create a new file:

vim list_bucket_content.rb

Paste the following content into the file:

#!/usr/bin/env ruby

load 'conn.rb'

new_bucket = AWS::S3::Bucket.find('my-new-bucket1')
new_bucket.each do |object|
        puts "#{object.key}\t#{object.about['content-length']}\t#{object.about['last-modified']}"
end

Save the file and exit the editor.

Make the file executable.
```
chmod +x list_bucket_content.rb
```

Run the file:

./list_bucket_content.rb

The output will look something like this:

hello.txt    12    Fri, 22 Jan 2016 15:54:52 GMT

6.7.2.7. Deleting a empty bucket

Create a new file:

vim del_empty_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Bucket.delete('my-new-bucket1')

Save the file and exit the editor.

Make the file executable:
```
chmod +x del_empty_bucket.rb
```
Run the file:
```
./del_empty_bucket.rb | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.
Note
Please edit the create_bucket.rb file to create empty buckets like my-new-bucket9, my-new-bucket10 etc and edit the above mentioned del_empty_bucket.rb file accordingly before trying to delete empty buckets.

6.7.2.8. Deleting a non-empty bucket (forcefully)

Create a new file:

vim del_non_empty_bucket.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::Bucket.delete('my-new-bucket1', :force => true)

Save the file and exit the editor.

Make the file executable:
```
chmod +x del_non_empty_bucket.rb
```
Run the file:
```
./del_non_empty_bucket.rb | echo $?
```
If the bucket is successfully deleted, the command will return 0 as output.

6.7.2.9. Deleting an object

Create a new file:

vim delete_object.rb

Paste the following contents into the file:

#!/usr/bin/env ruby

load 'conn.rb'

AWS::S3::S3Object.delete('hello.txt', 'my-new-bucket1')

Save the file and exit the editor.

Make the file executable:
```
chmod +x delete_object.rb
```
Run the file:
```
./delete_object.rb
```
This will delete the object hello.txt.

Language:

Language and Page Formatting Options

Language:

Red Hat Training

Chapter 6. Object Gateway S3 API

6.1. API

6.1.1. Feature Support

6.1.2. Unsupported Header Fields

6.2. Common

6.2.1. Bucket and Host Name

6.2.2. Common Request Headers

6.2.3. Common Response Status

6.3. Authentication and ACLs

6.3.1. Authentication

6.3.2. Access Control Lists (ACLs)

6.4. Service Operations

6.4.1. List Buckets

6.4.1.1. Syntax

6.4.1.2. Response Entities

6.5. Bucket Operations

6.5.1. PUT Bucket

6.5.1.1. Constraints

6.5.1.2. Syntax

6.5.1.3. Parameters

6.5.1.4. HTTP Response

6.5.2. DELETE Bucket

6.5.2.1. Syntax

6.5.2.2. HTTP Response

6.5.3. GET Bucket

6.5.3.1. Syntax

6.5.3.2. Parameters

6.5.3.3. HTTP Response

6.5.3.4. Bucket Response Entities

6.5.3.5. Object Response Entities

6.5.4. Get Bucket Location

6.5.4.1. Syntax

6.5.4.2. Response Entities

6.5.5. Get Bucket ACLs

6.5.5.1. Syntax

6.5.5.2. Response Entities

6.5.6. PUT Bucket ACLs

6.5.6.1. Syntax

6.5.6.2. Request Entities

6.5.7. List Bucket Object Versions

6.5.7.1. Syntax

6.5.7.2. Parameters

6.5.7.3. Response Entities

6.5.8. List Bucket Multipart Uploads

6.5.8.1. Syntax

6.5.8.2. Parameters

6.5.8.3. Response Entities

6.6. Object Operations

6.6.1. PUT Object

6.6.1.1. Syntax

6.6.1.2. Request Headers

6.6.1.3. Response Headers

6.6.2. Copy Object

6.6.2.1. Syntax

6.6.2.2. Request Headers

6.6.2.3. Response Entities

6.6.3. Remove Object

6.6.3.1. Syntax

6.6.4. Get Object

6.6.4.1. Syntax

6.6.4.2. Request Headers

6.6.4.3. Response Headers

6.6.5. Get Object Information

6.6.5.1. Syntax

6.6.5.2. Request Headers

6.6.5.3. Response Headers

6.6.6. Get Object ACL

6.6.6.1. Syntax

6.6.6.2. Response Headers

6.6.6.3. Response Entities

6.6.7. Set Object ACL

6.6.7.1. Syntax

6.6.7.2. Request Entities

6.6.8. Initiate Multipart Upload

6.6.8.1. Syntax

6.6.8.2. Request Headers