Chapter 5. Managing namespace buckets

Namespace buckets let you connect data repositories on different providers together, so you can interact with all of your data through a single unified view. Add the object bucket associated with each provider to the namespace bucket, and access your data through the namespace bucket to see all of your object buckets at once. This lets you write to your preferred storage provider while reading from multiple other storage providers, greatly reducing the cost of migrating to a new storage provider.

Note

A namespace bucket can only be used if its write target is available and functional.

5.1. Amazon S3 API endpoints for objects in namespace buckets

You can interact with objects in namespace buckets using the Amazon Simple Storage Service (S3) API.

Red Hat OpenShift Container Storage 4.6 onwards supports the following namespace bucket operations:

See the Amazon S3 API reference documentation for the most up-to-date information about these operations and how to use them.

Additional resources

5.2. Adding a namespace bucket using the Multicloud Object Gateway CLI and YAML

For more information about namespace buckets, see Managing namespace buckets.

Depending on the type of your deployment and whether you want to use YAML or the Multicloud Object Gateway CLI, choose one of the following procedures to add a namespace bucket:

5.2.1. Adding an AWS S3 namespace bucket using YAML

Prerequisites

Procedure

  1. Create a secret with the credentials: 

    apiVersion: v1
kind: Secret
metadata:
name: <namespacestore-secret-name>
type: Opaque
data:
AWS_ACCESS_KEY_ID: <AWS ACCESS KEY ID ENCODED IN BASE64>
AWS_SECRET_ACCESS_KEY: <AWS SECRET ACCESS KEY ENCODED IN BASE64>
    1. You must supply and encode your own AWS access key ID and secret access key using Base64, and use the results in place of <AWS ACCESS KEY ID ENCODED IN BASE64> and <AWS SECRET ACCESS KEY ENCODED IN BASE64>. ii. Replace <namespacestore-secret-name> with a unique name.

  2. Create a NamespaceStore resource using OpenShift Custom Resource Definitions (CRDs). A NamespaceStore represents underlying storage to be used as a read or write target for the data in the Multicloud Object Gateway namespace buckets. To create a NamespaceStore resource, apply the following YAML: 

    apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  finalizers:
  - noobaa.io/finalizer
  labels:
    app: noobaa
  name: <resource-name>
  namespace: openshift-storage
spec:
  awsS3:
    secret:
      name: <namespacestore-secret-name>
      namespace: <namespace-secret>
    targetBucket: <target-bucket>
  type: aws-s3
    1. Replace <resource-name> with the name you want to give to the resource.
    2. Replace <namespacestore-secret-name> with the secret created in step 1.
    3. Replace <namespace-secret> with the namespace where the secret can be found.
    4. Replace <target-bucket> with the target bucket you created for the NamespaceStore.

  3. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • A namespace policy of type single requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type:
    single:
      resource: <resource>

      Replace <my-bucket-class> with a unique namespace bucket class name.

      Replace <resource> with the name of a single namespace-store that will define the read and write target of the namespace bucket.

    • A namespace policy of type multi requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type: Multi
    multi:
      writeResource: <write-resource>
      readResources:
      - <read-resources>
      - <read-resources>

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <write-resource> with the name of a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources> with a list of the names of the namespace-stores that will define the read targets of the namespace bucket.

  4. Apply the following YAML to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2. 

    apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: <resource-name>
  namespace: openshift-storage
spec:
  generateBucketName: <my-bucket>
  storageClassName: noobaa.noobaa.io
  additionalConfig:
    bucketclass: <my-bucket-class>
    Note

    For IBM Power Systems and IBM Z infrastructure use storageClassName as openshift-storage.noobaa.io

    1. Replace <my-bucket-class> with the bucket class created in the previous step.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.2.2. Adding an IBM COS namespace bucket using YAML

Prerequisites

Procedure

  1. Create a secret with the credentials: 

    apiVersion: v1
kind: Secret
metadata:
name: <namespacestore-secret-name>
type: Opaque
data:
IBM_COS_ACCESS_KEY_ID: <IBM COS ACCESS KEY ID ENCODED IN BASE64>
IBM_COS_SECRET_ACCESS_KEY: <IBM COS SECRET ACCESS KEY ENCODED IN BASE64>
    1. You must supply and encode your own IBM COS access key ID and secret access key using Base64, and use the results in place of <IBM COS ACCESS KEY ID ENCODED IN BASE64> and `<IBM COS SECRET ACCESS KEY ENCODED IN BASE64>.
    2. Replace <namespacestore-secret-name> with a unique name.

  2. Create a NamespaceStore resource using OpenShift Custom Resource Definitions (CRDs). A NamespaceStore represents underlying storage to be used as a read or write target for the data in the Multicloud Object Gateway namespace buckets. To create a NamespaceStore resource, apply the following YAML: 

    apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  finalizers:
  - noobaa.io/finalizer
  labels:
    app: noobaa
  name: bs
  namespace: openshift-storage
spec:
  s3Compatible:
    endpoint: <IBM COS ENDPOINT>
    secret:
      name: <namespacestore-secret-name>
      namespace: <namespace-secret>
    signatureVersion: v2
    targetBucket: <target-bucket>
  type: ibm-cos
    1. Replace <IBM COS ENDPOINT> with the appropriate IBM COS endpoint.
    2. Replace <namespacestore-secret-name> with the secret created in step 1.
    3. Replace <namespace-secret> with the namespace where the secret can be found.
    4. Replace <target-bucket> with the target bucket you created for the NamespaceStore.

  3. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • A namespace policy of type single requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type:
    single:
      resource: <resource>

      Replace <my-bucket-class> with a unique namespace bucket class name.

      Replace <resource> with a the name of a single namespace-store that will define the read and write target of the namespace bucket.

    • A namespace policy of type multi requires the following configuration: 

      apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  labels:
    app: noobaa
  name: <my-bucket-class>
  namespace: openshift-storage
spec:
  namespacePolicy:
    type: Multi
    multi:
      writeResource: <write-resource>
      readResources:
      - <read-resources>
      - <read-resources>

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <write-resource> with the name of a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources> with a list of the names of namespace-stores that will define the read targets of the namespace bucket.

  4. Apply the following YAML to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2. 

    apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: <resource-name>
  namespace: openshift-storage
spec:
  generateBucketName: <my-bucket>
  storageClassName: noobaa.noobaa.io
  additionalConfig:
    bucketclass: <my-bucket-class>
    Note

    For IBM Power Systems and IBM Z infrastructure use storageClassName as openshift-storage.noobaa.io

    1. Replace <my-bucket-class> with the bucket class created in the previous step.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.2.3. Adding an AWS S3 namespace bucket using the Multicloud Object Gateway CLI

Prerequisites

  • A running OpenShift Container Storage Platform
  • Access to the Multicloud Object Gateway, see Chapter 2, Accessing the Multicloud Object Gateway with your applications

  • Download the Multicloud Object Gateway command-line interface: 

    # subscription-manager repos --enable=rh-ocs-4-for-rhel-8-x86_64-rpms
# yum install mcg
    Note

    Specify the appropriate architecture for enabling the repositories using subscription manager. For instance, in case of IBM Z infrastructure use the following command: 

    # subscription-manager repos --enable=rh-ocs-4-for-rhel-8-s390x-rpms

    Alternatively, you can install the mcg package from the OpenShift Container Storage RPMs found here https://access.redhat.com/downloads/content/547/ver=4/rhel---8/4/x86_64/package.

    Note

    Choose the correct Product Variant according to your architecture.

Procedure

  1. Create a NamespaceStore resource. A NamespaceStore represents an underlying storage to be used as a read or write target for the data in Multicloud Object Gateway namespace buckets. From the MCG command-line interface, run the following command: 

    noobaa namespacestore create aws-s3 <namespacestore > --access-key <AWS ACCESS KEY> --secret-key <AWS SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
    1. Replace <namespacestore> with the name of the NamespaceStore.
    2. Replace <AWS ACCESS KEY> and <AWS SECRET ACCESS KEY> with an AWS access key ID and secret access key you created for this purpose.
    3. Replace <bucket-name> with an existing AWS bucket name. This argument tells Multicloud Object Gateway which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.

  2. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • Run the following command to create a namespace bucket class with a namespace policy of type single: 

      noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage

      Replace <resource-name> with the name you want to give the resource.

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <resource> with a single namespace-store that will define the read and write target of the namespace bucket.

    • Run the following command to create a namespace bucket class with a namespace policy of type multi: 

      noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage

      Replace <resource-name> with the name you want to give the resource.

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <write-resource> with a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources> with a list of namespace-stores separated by commas that will define the read targets of the namespace bucket.

  3. Run the following command to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2. 

    noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
    1. Replace <bucket-name> with a bucket name of your choice.
    2. Replace <custom-bucket-class> with the name of the bucket class created in step 2.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.2.4. Adding an IBM COS namespace bucket using the Multicloud Object Gateway CLI

Prerequisites

  • A running OpenShift Container Storage Platform
  • Access to the Multicloud Object Gateway, see Chapter 2, Accessing the Multicloud Object Gateway with your applications

  • Download the Multicloud Object Gateway command-line interface: 

    # subscription-manager repos --enable=rh-ocs-4-for-rhel-8-x86_64-rpms
# yum install mcg
    Note

    Specify the appropriate architecture for enabling the repositories using subscription manager. For instance,

    • For IBM Power Systems, use the following command: 
    # subscription-manager repos --enable=rh-ocs-4-for-rhel-8-ppc64le-rpms
    • For IBM Z infrastructure, use the following command: 
    # subscription-manager repos --enable=rh-ocs-4-for-rhel-8-s390x-rpms

    Alternatively, you can install the mcg package from the OpenShift Container Storage RPMs found here https://access.redhat.com/downloads/content/547/ver=4/rhel---8/4/x86_64/package.

    Note

    Choose the correct Product Variant according to your architecture.

Procedure

  1. Create a NamespaceStore resource. A NamespaceStore represents an underlying storage to be used as a read or write target for the data in Multicloud Object Gateway namespace buckets. From the MCG command-line interface, run the following command: 

    noobaa namespacestore create ibm-cos <namespacestore> --endpoint <IBM COS ENDPOINT> --access-key <IBM ACCESS KEY> --secret-key <IBM SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
    1. Replace <namespacestore> with the name of the NamespaceStore.
    2. Replace <IBM ACCESS KEY>, <IBM SECRET ACCESS KEY>, <IBM COS ENDPOINT> with an IBM access key ID, secret access key and the appropriate regional endpoint that corresponds to the location of the existing IBM bucket.
    3. Replace <bucket-name> with an existing IBM bucket name. This argument tells Multicloud Object Gateway which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.

  2. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • Run the following command to create a namespace bucket class with a namespace policy of type single: 

      noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage

      Replace <resource-name> with the name you want to give the resource.

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <resource> with a single namespace-store that will define the read and write target of the namespace bucket.

    • Run the following command to create a namespace bucket class with a namespace policy of type multi: 

      noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage

      Replace <resource-name> with the name you want to give the resource.

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <write-resource> with a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources> with a list of namespace-stores separated by commas that will define the read targets of the namespace bucket.

  3. Run the following command to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2. 

    noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
    1. Replace <bucket-name> with a bucket name of your choice.
    2. Replace <custom-bucket-class> with the name of the bucket class created in step 2.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.3. Adding a namespace bucket using the OpenShift Container Platform user interface

With the release of OpenShift Container Storage 4.8, namespace buckets can be added using the OpenShift Container Platform user interface. For more information about namespace buckets, see Managing namespace buckets.

Prerequisites

  • Openshift Container Platform with OpenShift Container Storage operator installed
  • Access to the Multicloud Object Gateway.

Procedure

  1. Log into the OpenShift Web Console.
  2. Click Operators → Installed Operators from the left pane of the OpenShift Web Console to view the installed operators.
  3. Click OpenShift Container Storage Operator. If not visible, then move to all projects or openshift-storage namespace.
  4. Click the Namespace Store tab to create a namespacestore resources to be used in the namespace bucket.

  5. On the Namespace Store page, perform the following:

    1. Click Create namespace store.
    2. Enter a namespacestore name.
    3. Choose a provider.
    4. Choose a region.
    5. Either select an existing secret, or click Switch to credentials to create a secret by entering a secret key and secret access key.
    6. Choose a target bucket.
    7. Click Create.
    8. Verify the namespacestore is in the Ready state.
    9. Repeat until you have the desired amount of resources.
  6. Click the Bucket Class tab.
  7. Click Create a new Bucket Class.

  8. On the Create a new Bucket Class page, perform the following:

    1. Select the Namespace radio button.
    2. Enter a Bucket Class name.
    3. Add a description (optional).
    4. Click Next.
  9. On the next page, choose a namespace policy type for your namespace bucket, then click Next.

  10. Select the target resource(s).

    • If your namespace policy type is Single, you need to choose a read resource.
    • If your namespace policy type is Multi, you need to choose read resources and a write resource.
    • If your namespace policy type is Cache, you need to choose a Hub namespace store that defines the read and write target of the namespace bucket.
  11. Click Next.
  12. Review your new bucket class, then click Create Bucketclass.
  13. On the BucketClass page, verify that your newly created resource is in the Created phase.
  14. In the OpenShift Console, click Storage → Overview and click the Object tab.
  15. Click Multicloud Object Gateway.
  16. Click Buckets, and click on the Namespace Buckets tab.

  17. Click Create Namespace Bucket.

    1. On the Choose Name tab, specify a Name for the namespace bucket and click Next.

    2. On the Set Placement tab:

      1. Under Read Policy, select the checkbox for each namespace resource created in step 5 that the namespace bucket should read data from.
      2. If the namespace policy type you are using is Multi, then Under Write Policy, specify which namespace resource the namespace bucket should write data to.
      3. Click Next.
    3. Click Create.

Verification

  • Verify that the namespace bucket is listed with a green check mark in the State column, the expected number of read resources, and the expected write resource name.