Chapter 2. Creating an Amazon S3 client using notebook cells

To interact with data in Amazon S3 buckets, you must create a local client to handle requests to that service.

Prerequisites

  • Access to a JupyterHub notebook server running on Red Hat OpenShift Data Science.
  • Define values for the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables when you launch your notebook server, using the values from your Amazon Web Services account under My Security Credentials.

Procedure

  1. In a new notebook cell, import the required libraries by adding the following:

    import os
    import boto3
    from boto3 import session
  2. In another new notebook cell, define the following to create your session and client.

    1. Define your credentials.

      key_id = os.environ.get(AWS_ACCESS_KEY_ID)
      secret_key = os.environ.get(AWS_SECRET_ACCESS_KEY)
    2. Define the client session.

      session = boto3.session.Session(aws_access_key_id=key_id, aws_secret_access_key=secret_key)
    3. Define the client connection.

      s3_client = boto3.client(s3, aws_access_key_id=key_id, aws_secret_access_key=secret_key)

Verification

  • Create a new cell and run an Amazon S3 command such as the following:

    s3_client.list_buckets()

    A successful response includes a HTTPStatusCode of 200 and a list of Buckets similar to the following:

     'Buckets': [{'Name': 'my-app-asdf3-image-registry-us-east-1-wbmlcvbasdfasdgvtsmkpt',
       'CreationDate': datetime.datetime(2021, 4, 21, 6, 8, 52, tzinfo=tzlocal())},
      {'Name': 'cf-templates-18rxasdfggawsvb-us-east-1',
       'CreationDate': datetime.datetime(2021, 2, 15, 18, 35, 34, tzinfo=tzlocal())}