Menu Close

Chapter 5. Managing namespace buckets

Namespace buckets let you connect data repositories on different providers together, so you can interact with all of your data through a single unified view. Add the object bucket associated with each provider to the namespace bucket, and access your data through the namespace bucket to see all of your object buckets at once. This lets you write to your preferred storage provider while reading from multiple other storage providers, greatly reducing the cost of migrating to a new storage provider.

You can interact with objects in a namespace bucket using the S3 API. See S3 API endpoints for objects in namespace buckets for more information.

Note

A namespace bucket can only be used if its write target is available and functional.

5.1. Amazon S3 API endpoints for objects in namespace buckets

You can interact with objects in namespace buckets using the Amazon Simple Storage Service (S3) API.

Red Hat OpenShift Container Storage 4.6 onwards supports the following namespace bucket operations:

See the Amazon S3 API reference documentation for the most up-to-date information about these operations and how to use them.

5.2. Adding a namespace bucket using the Multicloud Object Gateway CLI and YAML

For more information about namespace buckets, see Managing namespace buckets.

Depending on the type of your deployment and whether you want to use YAML or the Multicloud Object Gateway CLI, choose one of the following procedures to add a namespace bucket:

5.2.1. Adding an AWS S3 namespace bucket using YAML

Prerequisites

Procedure

  1. Create a secret with the credentials:

    apiVersion: v1
    kind: Secret
    metadata:
    name: <namespacestore-secret-name>
    type: Opaque
    data:
    AWS_ACCESS_KEY_ID: <AWS ACCESS KEY ID ENCODED IN BASE64>
    AWS_SECRET_ACCESS_KEY: <AWS SECRET ACCESS KEY ENCODED IN BASE64>
    1. You must supply and encode your own AWS access key ID and secret access key using Base64, and use the results in place of <AWS ACCESS KEY ID ENCODED IN BASE64> and <AWS SECRET ACCESS KEY ENCODED IN BASE64>. ii. Replace <namespacestore-secret-name> with a unique name.
  2. Create a NamespaceStore resource using OpenShift Custom Resource Definitions (CRDs). A NamespaceStore represents underlying storage to be used as a read or write target for the data in the Multicloud Object Gateway namespace buckets. To create a NamespaceStore resource, apply the following YAML:

    apiVersion: noobaa.io/v1alpha1
    kind: NamespaceStore
    metadata:
      finalizers:
      - noobaa.io/finalizer
      labels:
        app: noobaa
      name: mybucketnamespace
      namespace: k8snamespace
    spec:
      awsS3:
        secret:
          name: <namespacestore-secret-name>
          namespace: k8snamespace
        targetBucket: awsdatalake
      type: aws-s3
    1. Replace <namespacestore-secret-name> with with the secret created in step 1.
  3. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • A namespace policy of type single requires the following configuration:

      apiVersion: noobaa.io/v1alpha1
      kind: BucketClass
      metadata:
        labels:
          app: noobaa
        name: <my-bucket-class>
        namespace: openshift-storage
      spec:
        namespacePolicy:
          type:
          single:
            resource: <resource>

      Replace <my-bucket-class> with a unique namespace bucket class name.

      Replace <resource> with a single namespace-store that will define the read and write target of the namespace bucket.

    • A namespace policy of type multi requires the following configuration:

      apiVersion: noobaa.io/v1alpha1
      kind: BucketClass
      metadata:
        labels:
          app: noobaa
        name: <my-bucket-class>
        namespace: openshift-storage
      spec:
        namespacePolicy:
          type: Multi
          multi:
            writeResource: <write-resource>
            readResources:
            - <read-resources>
            - <read-resources>

      Replace <my-bucket-class> with a unique bucket class name.

      Replace write-resource with a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources with a list of namespace-stores that will define the read targets of the namespace bucket.

  4. Apply the following YAML to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.

    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: my-bucket-claim
      namespace: my-app
    spec:
      generateBucketName: my-bucket
      storageClassName: noobaa.noobaa.io
      additionalConfig:
        bucketclass: <my-bucket-class>
    1. Replace <my-bucket-class> with the bucket class created in the previous step.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.2.2. Adding an IBM COS namespace bucket using YAML

Prerequisites

Procedure

  1. Create a secret with the credentials:

    apiVersion: v1
    kind: Secret
    metadata:
    name: <namespacestore-secret-name>
    type: Opaque
    data:
    IBM_COS_ACCESS_KEY_ID: <IBM COS ACCESS KEY ID ENCODED IN BASE64>
    IBM_COS_SECRET_ACCESS_KEY: <IBM COS SECRET ACCESS KEY ENCODED IN BASE64>
    1. You must supply and encode your own IBM COS access key ID and secret access key using Base64, and use the results in place of <IBM COS ACCESS KEY ID ENCODED IN BASE64> and `<IBM COS SECRET ACCESS KEY ENCODED IN BASE64>.
    2. Replace <namespacestore-secret-name> with a unique name.
  2. Create a NamespaceStore resource using OpenShift Custom Resource Definitions (CRDs). A NamespaceStore represents underlying storage to be used as a read or write target for the data in the Multicloud Object Gateway namespace buckets. To create a NamespaceStore resource, apply the following YAML:

    apiVersion: noobaa.io/v1alpha1
    kind: NamespaceStore
    metadata:
      finalizers:
      - noobaa.io/finalizer
      labels:
        app: noobaa
      name: bs
      namespace: k8snamespace
    spec:
      s3Compatible:
        endpoint: <IBM COS ENDPOINT>
        secret:
          name: <namespacestore-secret-name>
          namespace: openshift-storage
        signatureVersion: v2
        targetBucket: BUCKET
      type: ibm-cos
    1. Replace <IBM COS ENDPOINT> with the appropriate IBM COS endpoint.
    2. Replace <namespacestore-secret-name> with the secret created in step 1.
  3. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • A namespace policy of type single requires the following configuration:

      apiVersion: noobaa.io/v1alpha1
      kind: BucketClass
      metadata:
        labels:
          app: noobaa
        name: <my-bucket-class>
        namespace: openshift-storage
      spec:
        namespacePolicy:
          type:
          single:
            resource: <resource>

      Replace <my-bucket-class> with a unique namespace bucket class name.

      Replace <resource> with a single namespace-store that will define the read and write target of the namespace bucket.

    • A namespace policy of type multi requires the following configuration:

      apiVersion: noobaa.io/v1alpha1
      kind: BucketClass
      metadata:
        labels:
          app: noobaa
        name: <my-bucket-class>
        namespace: openshift-storage
      spec:
        namespacePolicy:
          type: Multi
          multi:
            writeResource: <write-resource>
            readResources:
            - <read-resources>
            - <read-resources>

      Replace <my-bucket-class> with a unique bucket class name.

      Replace write-resource with a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources with a list of namespace-stores that will define the read targets of the namespace bucket.

  4. Apply the following YAML to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.

    apiVersion: objectbucket.io/v1alpha1
    kind: ObjectBucketClaim
    metadata:
      name: my-bucket-claim
      namespace: my-app
    spec:
      generateBucketName: my-bucket
      storageClassName: noobaa.noobaa.io
      additionalConfig:
        bucketclass: <my-bucket-class>
    1. Replace <my-bucket-class> with the bucket class created in the previous step.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.2.3. Adding an AWS S3 namespace bucket using the Multicloud Object Gateway CLI

Prerequisites

Procedure

  1. Create a NamespaceStore resource. A NamespaceStore represents an underlying storage to be used as a read or write target for the data in Multicloud Object Gateway namespace buckets. From the MCG command-line interface, run the following command:

    noobaa namespacestore create aws-s3 <namespacestore> --access-key <AWS ACCESS KEY> --secret-key <AWS SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
    1. Replace <namespacestore> with the name of the NamespaceStore.
    2. Replace <AWS ACCESS KEY> and <AWS SECRET ACCESS KEY> with an AWS access key ID and secret access key you created for this purpose.
    3. Replace <bucket-name> with an existing AWS bucket name. This argument tells Multicloud Object Gateway which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.
  2. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • Run the following command to create a namespace bucket class with a namespace policy of type single:

      noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <resource> with a single namespace-store that will define the read and write target of the namespace bucket.

    • Run the following command to create a namespace bucket class with a namespace policy of type multi:

      noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage

      Replace <my-bucket-class> with a unique bucket class name.

      Replace write-resource with a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources with a list of namespace-stores separated by commas that will define the read targets of the namespace bucket.

  3. Run the following command to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.

    noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
    1. Replace <custom-bucket-class> with the name of the bucket class created in step 2.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.

5.2.4. Adding an IBM COS namespace bucket using the Multicloud Object Gateway CLI

Prerequisites

Procedure

  1. Create a NamespaceStore resource. A NamespaceStore represents an underlying storage to be used as a read or write target for the data in Multicloud Object Gateway namespace buckets. From the MCG command-line interface, run the following command:

    noobaa namespacestore create ibm-cos <namespacestore> --endpoint <IBM COS ENDPOINT> --access-key <IBM ACCESS KEY> --secret-key <IBM SECRET ACCESS KEY> --target-bucket <bucket-name> -n openshift-storage
    1. Replace <namespacestore> with the name of the NamespaceStore.
    2. Replace <IBM ACCESS KEY>, <IBM SECRET ACCESS KEY>, <IBM COS ENDPOINT> with an IBM access key ID, secret access key and the appropriate regional endpoint that corresponds to the location of the existing IBM bucket.
    3. Replace <bucket-name> with an existing IBM bucket name. This argument tells Multicloud Object Gateway which bucket to use as a target bucket for its backing store, and subsequently, data storage and administration.
  2. Create a namespace bucket class that defines a namespace policy for the namespace buckets. The namespace policy requires a type of either single or multi.

    • Run the following command to create a namespace bucket class with a namespace policy of type single:

      noobaa bucketclass create namespace-bucketclass single <my-bucket-class> --resource <resource> -n openshift-storage

      Replace <my-bucket-class> with a unique bucket class name.

      Replace <resource> with a single namespace-store that will define the read and write target of the namespace bucket.

    • Run the following command to create a namespace bucket class with a namespace policy of type multi:

      noobaa bucketclass create namespace-bucketclass multi <my-bucket-class> --write-resource <write-resource> --read-resources <read-resources> -n openshift-storage

      Replace <my-bucket-class> with a unique bucket class name.

      Replace write-resource with a single namespace-store that will define the write target of the namespace bucket.

      Replace <read-resources with a list of namespace-stores separated by commas that will define the read targets of the namespace bucket.

  3. Run the following command to create a bucket using an Object Bucket Class (OBC) resource that uses the bucket class defined in step 2.

    noobaa obc create my-bucket-claim -n openshift-storage --app-namespace my-app --bucketclass <custom-bucket-class>
    1. Replace <custom-bucket-class> with the name of the bucket class created in step 2.

Once the OBC is provisioned by the operator, a bucket is created in the Multicloud Object Gateway, and the operator creates a Secret and ConfigMap with the same name of the OBC on the same namespace of the OBC.