Menu Close
Chapter 5. Managing namespace buckets
Namespace buckets let you connect data repositories on different providers together, so you can interact with all of your data through a single unified view. Add the object bucket associated with each provider to the namespace bucket, and access your data through the namespace bucket to see all of your object buckets at once. This lets you write to your preferred storage provider while reading from multiple other storage providers, greatly reducing the cost of migrating to a new storage provider.
You can interact with objects in a namespace bucket using the S3 API. See S3 API endpoints for objects in namespace buckets for more information.
A namespace bucket can only be used if its write target is available and functional.
5.1. Amazon S3 API endpoints for objects in namespace buckets
You can interact with objects in namespace buckets using the Amazon Simple Storage Service (S3) API.
Red Hat OpenShift Container Storage 4.6 onwards supports the following namespace bucket operations:
See the Amazon S3 API reference documentation for the most up-to-date information about these operations and how to use them.
Additional resources
5.2. Adding a namespace bucket using the Multicloud Object Gateway CLI and YAML
For more information about namespace buckets, see Managing namespace buckets.
Depending on the type of your deployment and whether you want to use YAML or the Multicloud Object Gateway CLI, choose one of the following procedures to add a namespace bucket:
5.2.1. Adding an AWS S3 namespace bucket using YAML
Prerequisites
- A running OpenShift Container Storage Platform
- Access to the Multicloud Object Gateway, see Chapter 2, Accessing the Multicloud Object Gateway with your applications
Procedure
Create a secret with the credentials:
apiVersion: v1 kind: Secret metadata: name: <namespacestore-secret-name> type: Opaque data: AWS_ACCESS_KEY_ID: <AWS ACCESS KEY ID ENCODED IN BASE64> AWS_SECRET_ACCESS_KEY: <AWS SECRET ACCESS KEY ENCODED IN BASE64>
-
You must supply and encode your own AWS access key ID and secret access key using Base64, and use the results in place of
<AWS ACCESS KEY ID ENCODED IN BASE64>
and<AWS SECRET ACCESS KEY ENCODED IN BASE64>
. ii. Replace<namespacestore-secret-name>
with a unique name.
-
You must supply and encode your own AWS access key ID and secret access key using Base64, and use the results in place of
Create a NamespaceStore resource using OpenShift Custom Resource Definitions (CRDs). A NamespaceStore represents underlying storage to be used as a read or write target for the data in the Multicloud Object Gateway namespace buckets. To create a NamespaceStore resource, apply the following YAML:
apiVersion: noobaa.io/v1alpha1 kind: NamespaceStore metadata: finalizers: - noobaa.io/finalizer labels: app: noobaa name: mybucketnamespace namespace: k8snamespace spec: awsS3: secret: name: <namespacestore-secret-name> namespace: k8snamespace targetBucket: awsdatalake type: aws-s3
-
Replace
<namespacestore-secret-name>
with with the secret created in step 1.
-
Replace
Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either
single
ormulti
.A namespace policy of type
single
requires the following configuration:apiVersion: noobaa.io/v1alpha1 kind: BucketClass metadata: labels: app: noobaa name: <my-bucket-class> namespace: openshift-storage spec: namespacePolicy: type: single: resource: <resource>
Replace
<my-bucket-class>
with a unique namespace bucket class name.Replace
<resource>
with a single namespace-store that will define the read and write target of the namespace bucket.A namespace policy of type
multi
requires the following configuration:apiVersion: noobaa.io/v1alpha1 kind: BucketClass metadata: labels: app: noobaa name: <my-bucket-class> namespace: openshift-storage spec: namespacePolicy: type: Multi multi: writeResource: <write-resource> readResources: - <read-resources> - <read-resources>
Replace
<my-bucket-class>
with a unique bucket class name.Replace
write-resource
with a single namespace-store that will define the write target of the namespace bucket.Replace
<read-resources
with a list of namespace-stores that will define the read targets of the namespace bucket.
Apply the following YAML to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.
apiVersion: objectbucket.io/v1alpha1 kind: ObjectBucketClaim metadata: name: my-bucket-claim namespace: my-app spec: generateBucketName: my-bucket storageClassName: noobaa.noobaa.io additionalConfig: bucketclass: <my-bucket-class>
-
Replace
<my-bucket-class>
with the bucket class created in the previous step.
-
Replace
Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.
5.2.2. Adding an IBM COS namespace bucket using YAML
Prerequisites
- A running OpenShift Container Storage Platform
- Access to the Multicloud Object Gateway, see Chapter 2, Accessing the Multicloud Object Gateway with your applications
Procedure
Create a secret with the credentials:
apiVersion: v1 kind: Secret metadata: name: <namespacestore-secret-name> type: Opaque data: IBM_COS_ACCESS_KEY_ID: <IBM COS ACCESS KEY ID ENCODED IN BASE64> IBM_COS_SECRET_ACCESS_KEY: <IBM COS SECRET ACCESS KEY ENCODED IN BASE64>
-
You must supply and encode your own IBM COS access key ID and secret access key using Base64, and use the results in place of
<IBM COS ACCESS KEY ID ENCODED IN BASE64>
and `<IBM COS SECRET ACCESS KEY ENCODED IN BASE64>. -
Replace
<namespacestore-secret-name>
with a unique name.
-
You must supply and encode your own IBM COS access key ID and secret access key using Base64, and use the results in place of
Create a NamespaceStore resource using OpenShift Custom Resource Definitions (CRDs). A NamespaceStore represents underlying storage to be used as a read or write target for the data in the Multicloud Object Gateway namespace buckets. To create a NamespaceStore resource, apply the following YAML:
apiVersion: noobaa.io/v1alpha1 kind: NamespaceStore metadata: finalizers: - noobaa.io/finalizer labels: app: noobaa name: bs namespace: k8snamespace spec: s3Compatible: endpoint: <IBM COS ENDPOINT> secret: name: <namespacestore-secret-name> namespace: openshift-storage signatureVersion: v2 targetBucket: BUCKET type: ibm-cos
-
Replace
<IBM COS ENDPOINT>
with the appropriate IBM COS endpoint. -
Replace
<namespacestore-secret-name>
with the secret created in step 1.
-
Replace
Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either
single
ormulti
.A namespace policy of type
single
requires the following configuration:apiVersion: noobaa.io/v1alpha1 kind: BucketClass metadata: labels: app: noobaa name: <my-bucket-class> namespace: openshift-storage spec: namespacePolicy: type: single: resource: <resource>
Replace
<my-bucket-class>
with a unique namespace bucket class name.Replace
<resource>
with a single namespace-store that will define the read and write target of the namespace bucket.A namespace policy of type
multi
requires the following configuration:apiVersion: noobaa.io/v1alpha1 kind: BucketClass metadata: labels: app: noobaa name: <my-bucket-class> namespace: openshift-storage spec: namespacePolicy: type: Multi multi: writeResource: <write-resource> readResources: - <read-resources> - <read-resources>
Replace
<my-bucket-class>
with a unique bucket class name.Replace
write-resource
with a single namespace-store that will define the write target of the namespace bucket.Replace
<read-resources
with a list of namespace-stores that will define the read targets of the namespace bucket.
Apply the following YAML to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.
apiVersion: objectbucket.io/v1alpha1 kind: ObjectBucketClaim metadata: name: my-bucket-claim namespace: my-app spec: generateBucketName: my-bucket storageClassName: noobaa.noobaa.io additionalConfig: bucketclass: <my-bucket-class>
-
Replace
<my-bucket-class>
with the bucket class created in the previous step.
-
Replace
Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.
5.2.3. Adding an AWS S3 namespace bucket using the Multicloud Object Gateway CLI
Prerequisites
- A running OpenShift Container Storage Platform
- Access to the Multicloud Object Gateway, see Chapter 2, Accessing the Multicloud Object Gateway with your applications
Download the Multicloud Object Gateway command-line interface:
# subscription-manager repos --enable=rh-ocs-4-for-rhel-8-x86_64-rpms # yum install mcg
Alternatively, you can install the mcg package from the OpenShift Container Storage RPMs found here https://access.redhat.com/downloads/content/547/ver=4/rhel---8/4/x86_64/package.
Procedure
Create a NamespaceStore resource. A NamespaceStore represents an underlying storage to be used as a read or write target for the data in Multicloud Object Gateway namespace buckets. From the MCG command-line interface, run the following command:
noobaa namespacestore create aws-s3 <namespacestore> --access-key <AWS ACCESS KEY> --secret-key <AWS SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
-
Replace
<namespacestore>
with the name of the NamespaceStore. -
Replace
<AWS ACCESS KEY>
and<AWS SECRET ACCESS KEY>
with an AWS access key ID and secret access key you created for this purpose. -
Replace
<bucket-name>
with an existing AWS bucket name. This argument tells Multicloud Object Gateway which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.
-
Replace
Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either
single
ormulti
.Run the following command to create a namespace bucket class with a namespace policy of type
single
:noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage
Replace
<my-bucket-class>
with a unique bucket class name.Replace
<resource>
with a single namespace-store that will define the read and write target of the namespace bucket.Run the following command to create a namespace bucket class with a namespace policy of type
multi
:noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage
Replace
<my-bucket-class>
with a unique bucket class name.Replace
write-resource
with a single namespace-store that will define the write target of the namespace bucket.Replace
<read-resources
with a list of namespace-stores separated by commas that will define the read targets of the namespace bucket.
Run the following command to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.
noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
-
Replace
<custom-bucket-class>
with the name of the bucket class created in step 2.
-
Replace
Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.
5.2.4. Adding an IBM COS namespace bucket using the Multicloud Object Gateway CLI
Prerequisites
- A running OpenShift Container Storage Platform
- Access to the Multicloud Object Gateway, see Chapter 2, Accessing the Multicloud Object Gateway with your applications
Download the Multicloud Object Gateway command-line interface:
# subscription-manager repos --enable=rh-ocs-4-for-rhel-8-x86_64-rpms # yum install mcg
Alternatively, you can install the mcg package from the OpenShift Container Storage RPMs found here https://access.redhat.com/downloads/content/547/ver=4/rhel---8/4/x86_64/package.
Procedure
Create a NamespaceStore resource. A NamespaceStore represents an underlying storage to be used as a read or write target for the data in Multicloud Object Gateway namespace buckets. From the MCG command-line interface, run the following command:
noobaa namespacestore create ibm-cos <namespacestore> --endpoint <IBM COS ENDPOINT> --access-key <IBM ACCESS KEY> --secret-key <IBM SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
-
Replace
<namespacestore>
with the name of the NamespaceStore. -
Replace
<IBM ACCESS KEY>
,<IBM SECRET ACCESS KEY>
,<IBM COS ENDPOINT>
with an IBM access key ID, secret access key and the appropriate regional endpoint that corresponds to the location of the existing IBM bucket. -
Replace
<bucket-name>
with an existing IBM bucket name. This argument tells Multicloud Object Gateway which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.
-
Replace
Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either
single
ormulti
.Run the following command to create a namespace bucket class with a namespace policy of type
single
:noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage
Replace
<my-bucket-class>
with a unique bucket class name.Replace
<resource>
with a single namespace-store that will define the read and write target of the namespace bucket.Run the following command to create a namespace bucket class with a namespace policy of type
multi
:noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage
Replace
<my-bucket-class>
with a unique bucket class name.Replace
write-resource
with a single namespace-store that will define the write target of the namespace bucket.Replace
<read-resources
with a list of namespace-stores separated by commas that will define the read targets of the namespace bucket.
Run the following command to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.
noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
-
Replace
<custom-bucket-class>
with the name of the bucket class created in step 2.
-
Replace
Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.