Procedure

1. In a new notebook cell, import the required libraries by adding the following:

   import os
   import boto3
   from boto3 import session

2. In another new notebook cell, define the following to create your session and client.

   1. Define your credentials.

      key_id = os.environ.get('AWS_ACCESS_KEY_ID')
      secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')

   2. Define the client session.

      session = boto3.session.Session(aws_access_key_id=key_id, aws_secret_access_key=secret_key)

   3. Define the client connection.

      s3_client = boto3.client('s3', aws_access_key_id=key_id, aws_secret_access_key=secret_key)

Procedure

1. Create a new notebook cell and use the s3_client to list available buckets.

   s3_client.list_buckets()

2. You can make this list of buckets easier to read by only printing the name, rather than the full response, for example:

   for bucket in s3_client.list_buckets()['Buckets']:
       print(bucket['Name'])

   This returns output similar to the following:

   my-app-asdf3-image-registry-us-east-1-wbmlcvbasdgasdgtkpt
   cf-templates-18rxuasgasgvb-us-east-1

Procedure

1. Create a new notebook cell and list the objects in the bucket. For example:

   bucket_name = 'std-user-bucket1'
   s3_client.list_objects_v2(Bucket=bucket_name)

   This returns several objects in the following format:

   {'Key': 'docker/registry/v2/blobs/sha256/00/0080913dd3f10aadb34asfgsgsdgasdga072049c93606b98bec84adb259b424f/data',
   'LastModified': datetime.datetime(2021, 4, 22, 1, 26, 1, tzinfo=tzlocal()),
   'ETag': '"6e02fad2deassadfsf900a4bd7344ffe"',
   'Size': 4052,
   'StorageClass': 'STANDARD'}

2. You can make this list easier to read by printing only the key rather than the full response, for example:

   bucket_name = 'std-user-bucket1'
   for key in s3_client.list_objects_v2(Bucket=bucket_name)['Contents']:
       print(key['Key'])

   This returns output similar to the following:

   docker/registry/v2/blobs/sha256/00/0080913dd3f10aadb34asfgsgsdgasdga072049c93606b98bec84adb259b424f/data

3. You can also filter your query to list for a specific "path" or file name, for example:

   bucket_name = 'std-user-bucket1'
   for key in s3_client.list_objects_v2(Bucket=bucket_name,Prefix='<start_of_file_path>')['Contents']:
       print(key['Key'])

   In the preceding example, replace <start_of_file_path> with your own value.

Procedure

1. Define the following details in a notebook cell:

   1. The bucket that the file is in. Replace <name_of_the_bucket> with your own value.

      bucket_name = '<name_of_the_bucket>'

   2. The name of the file to download. Replace <name_of_the_file_to_download> with your own value.

      file_name = '<name_of_the_file_to_download>' # Full path from the bucket

   3. The name that you want the file to have after it is downloaded. This can be a full path, a relative path, or just a new file name. Replace <name_of_the_file_when_downloaded> with your own value.

      new_file_name = '<name_of_the_file_when_downloaded>'

2. Download the file, specifying the previous variables as arguments.

   s3_client.download_file(bucket_name, file_name, new_file_name)

   Note

   If you want to retrieve a file as an object that you can then stream as a standard file using the read() method, refer to the Amazon Web Services get object command reference.

Procedure

1. Define the following details in a notebook cell:

   1. The name of the file to upload. This must include the full local path to the file. Replace <name_of_the_file_to_upload> with your own value.

      file_name = '<name_of_the_file_to_upload>'

   2. The name of the bucket to upload the file to. Replace <name_of_the_bucket> with your own value.

      bucket_name = '<name_of_the_bucket>'

   3. The full key to use to save the file to the bucket. Replace <full_path_and_file_name> with your own value.

      key = '<full_path_and_file_name>'

2. Upload the file, specifying the previous variables as arguments.

   s3_client.upload_file(file_name, bucket_name, key)