Chapter 10. Upgrading

You can upgrade your existing Ansible Automation Platform deployment to the newer version. The upgrade process covers upgrading the automation hub, automation controller, and extension nodes. The upgrade process takes roughly the same amount of time as a Ansible Automation Platform deployment install. You are required to create a backup before running the upgrade.

Note

Ansible Automation Platform from AWS Marketplace supports sequential upgrades. All upgrades should be no more than one major version behind the version you are currently upgrading to. For example, to upgrade Ansible Automation Platform to 2.4.20230630, you must be on version 2.3.20230221.

Prerequisites

  • Docker must be installed to run the upgrade playbook.
  • A Linux or macOS system, where the ansible-on-clouds-ops container image will run.
  • The upgrade process requires several volumes to be mounted. Prepare a fresh directory to be used for this process.

The following procedures form the upgrade process:

  1. Backup your Ansible Automation Platform 2.3 stack

  2. Upgrade Ansible Automation Platform

    1. Pull the ansible-on-clouds-ops 2.4 container image
    2. Ensure minimum required permissions are met
    3. Generate the data file
    4. Update the data file
    5. Begin upgrading Ansible Automation Platform by running the operational container
  3. (Optional) Restore the stack from a backup.

10.1. Backup before upgrade

Important

Before beginning the upgrade of your Ansible Automation Platform environment, it is first required to take a backup of your environment at its current version.

To backup your Ansible Automation Platform environment at the earlier version, follow the Ansible on Clouds 2.3 backup instructions.

10.2. Upgrade Ansible Automation Platform

10.2.1. Pulling the ansible-on-clouds-ops container image

Procedure

  • Pull the Docker image for Ansible on Clouds operational container with the same tag as the version you want to upgrade to.

    Note

    Before pulling the docker image, ensure you are logged in to registry.redhat.io using docker. Use the following command to login to registry.redhat.io.

    $ docker login registry.redhat.io

    For more information about registry login, see Registry Authentication

    Note

    The Ansible on Clouds operational image tag must match the version that you want to upgrade to. For example, if your foundation deployment version is 2.3.20230221, pull the operational image with tag 2.4.20230630 to upgrade to version 2.4.20230630.

    $ export IMAGE=registry.redhat.io/ansible-on-clouds/ansible-on-clouds-ops-rhel9:2.4.20230630
    $ docker pull $IMAGE --platform=linux/amd64

10.2.2. Required permissions

You must have the following AWS IAM permissions to upgrade the stack:

required-roles:
  ec2:
    actions:
      - ec2:DescribeInstances
      - ec2:GetLaunchTemplateData
      - ec2:DescribeAccountAttributes
      - ec2:DescribeLaunchTemplates
      - ec2:CreateTags
      - ec2:RunInstances
      - ec2:DescribeInstanceTypes
      - ec2:DescribeSubnets
      - ec2:DeleteTags
      - ec2:DescribeRegions
      - ec2:DescribeAvailabilityZones
      - ec2:DeleteLaunchTemplate
      - ec2:DescribeTags
      - ec2:DescribeLaunchTemplateVersions
      - ec2:DescribeSecurityGroups
      - ec2:CreateLaunchTemplateVersion
      - ec2:CreateLaunchTemplate
      - ec2:DescribeVpcs
      - ec2:ModifyLaunchTemplate
    resources:
      - "*"
  iam:
    actions:
      - iam:ListRoleTags
      - iam:AttachRolePolicy
      - iam:PutRolePolicy
      - iam:AddRoleToInstanceProfile
      - iam:ListAttachedRolePolicies
      - iam:GetRole
      - iam:GetRolePolicy
      - iam:CreateInstanceProfile
      - iam:ListInstanceProfilesForRole
      - iam:PassRole
      - iam:GetInstanceProfile
      - iam:CreateServiceLinkedRole
    resources:
      - "*"
  secretsmanager:
    actions:
      - secretsmanager:DescribeSecret
      - secretsmanager:ListSecrets
      - secretsmanager:TagResource
      - secretsmanager:UntagResource
      - secretsmanager:CreateSecret
      - secretsmanager:GetSecretValue
      - secretsmanager:UpdateSecret
      - secretsmanager:GetResourcePolicy
    resources:
      - "*"
  ssm:
    actions:
      - ssm:StartSession
      - ssm:TerminateSession
    resources:
      - "*"
  autoscaling:
    actions:
      - autoscaling:DescribeAutoScalingGroups
      - autoscaling:UpdateAutoScalingGroup
      - autoscaling:DescribeInstanceRefreshes
      - autoscaling:DeleteTags
      - autoscaling:DescribeTags
      - autoscaling:CreateOrUpdateTags
      - autoscaling:CreateAutoScalingGroup
      - autoscaling:DescribeLaunchConfigurations
      - autoscaling:DescribeLifecycleHooks
      - autoscaling:StartInstanceRefresh
      - autoscaling:CreateLaunchConfiguration
      - autoscaling:DisableMetricsCollection
      - autoscaling:DeleteLaunchConfiguration
      - autoscaling:DetachLoadBalancerTargetGroups
      - autoscaling:AttachLoadBalancerTargetGroups
    resources:
      - "*"
  s3:
    actions:
      - s3:DeleteObject
      - s3:PutObject
      - s3:GetObject
      - s3:GetBucketLocation
    resources:
      - "*"
  cloudformation:
    actions:
      - cloudformation:ListStackResources
      - cloudformation:DescribeStacks
      - cloudformation:GetTemplate
    resources:
      - "*"
  elasticloadbalancing:
    actions:
      - elasticloadbalancing:DescribeTargetGroupAttributes
      - elasticloadbalancing:DescribeTags
      - elasticloadbalancing:DescribeTargetHealth
      - elasticloadbalancing:DescribeTargetGroups
      - elasticloadbalancing:ModifyTargetGroup
      - elasticloadbalancing:DescribeLoadBalancers
      - elasticloadbalancing:DescribeLoadBalancerAttributes
      - elasticloadbalancing:DescribeListeners
      - elasticloadbalancing:CreateListener
      - elasticloadbalancing:DeleteListener
      - elasticloadbalancing:DescribeRules
      - elasticloadbalancing:DeleteTargetGroup
      - elasticloadbalancing:CreateTargetGroup
      - elasticloadbalancing:ModifyTargetGroupAttributes
      - elasticloadbalancing:AddTags
      - elasticloadbalancing:RemoveTags
    resources:
      - "*"

10.2.3. Generating the data file

The commands in this section create a directory and populate it with an empty data template that, when populated, is used during the upgrade.

Procedure

  1. Run the following commands to generate the required data file.

    # Create a folder to hold the configuration files
    $ mkdir command_generator_data
  2. Populate the command_generator_data folder with the configuration file template.

    Note

    On Linux, any file or directory created by the command generator is owned by root:root by default. To change the ownership of the files and directories, you can run the sudo chmod command after the files are created. For more information, read Command generator - Linux files owned by root.

    $ docker run --rm -v $(pwd)/command_generator_data:/data $IMAGE \
    command_generator_vars aws_upgrade \
    --output-data-file /data/extra_vars.yml
  3. After running these commands, a command_generator_data/extra_vars.yml template file is created. This template file resembles the following:

    aws_upgrade:
      ansible_config_path:
      cloud_credentials_path:
      deployment_name:
      extra_vars:
        aws_backup_taken:
        aws_region:
        aws_ssm_bucket_name:
        seller_name:

10.2.4. Updating the data file

You must populate the data file before triggering the upgrade. Each parameter is required unless it is noted as optional. If the parameter is noted as optional, both its key and value can be removed from the data file.

The following variables are parameters listed in the data file:

  • ansible_config_path (Optional) Only use if overriding with a custom ansible_config.
  • cloud_credentials_path is the path to your AWS credentials file.
  • deployment_name is the name of the foundation deployment.

    • This is the same name you used when you deployed the foundation.
  • aws_backup_taken is the verification that a manual backup of the current deployment was recently created prior to running this upgrade.
  • aws_region is the AWS region where the foundation deployment is located.
  • aws_ssm_bucket_name is the name of the S3 bucket where temporary configuration files for the AWS SSM are stored. You can use an existing bucket or create a new one.

    Note

    The aws_ssm_bucket_name parameter is ONLY used to store temporary config files. It does NOT need to be saved for use in other playbooks. Any valid existing bucket can be used. For more information on creating S3 buckets, read AWS Creating A Bucket in the AWS documentation.

    The bucket name must not contain uppper case letters.

  • seller_name (Optional) is used to designate the AWS Marketplace Seller. The default value is redhatinc. If you are based in EMEA and purchased this offering through the redhatlimited AWS Marketplace seller, ensure that this value is set to redhatlimited.

After populating the data file, it should resemble the following.

The following values are provided as examples:

aws_upgrade:
  ansible_config_path:
  cloud_credentials_path: ~/.aws/credentials
  deployment_name: AnsibleAutomationPlatform
  extra_vars:
    aws_backup_taken: true
    aws_region: us-east-1
    aws_ssm_bucket_name: aap-ssm-bucket
    seller_name: redhatinc

10.2.5. Running the upgrade playbook

Note

Ansible Automation Platform upgrade to 2.4.20230630 updates the listeners on its internal load balancers. If a resource dependancy was added after installation, this must be temporarily removed to allow the upgrade to succeed. See technical note for more detail.

  1. To run the upgrade, run the command generator to generate the upgrade CLI command:

    $ docker run --rm -v $(pwd)/command_generator_data:/data $IMAGE command_generator --data-file /data/extra_vars.yml

    This generates the following upgrade command:

    -----------------------------------------------
    docker run --rm --env PLATFORM=AWS -v ~/.aws/credentials:/home/runner/.aws/credentials:ro --env ANSIBLE_CONFIG=../aws-ansible.cfg --env DEPLOYMENT_NAME=AnsibleAutomationPlatform --env GENERATE_INVENTORY=true  $IMAGE redhat.ansible_on_clouds.aws_upgrade -e 'aws_foundation_stack_name=AnsibleAutomationPlatform aws_region=us-east-1 aws_ssm_bucket_name=aap-ssm-bucket aws_backup_taken=True seller_name=redhatinc'
    ===============================================
  2. Run the given upgrade command to trigger the upgrade.

    $ docker run --rm --env PLATFORM=AWS -v ~/.aws/credentials:/home/runner/.aws/credentials:ro --env ANSIBLE_CONFIG=../aws-ansible.cfg --env DEPLOYMENT_NAME=AnsibleAutomationPlatform --env GENERATE_INVENTORY=true  $IMAGE redhat.ansible_on_clouds.aws_upgrade -e 'aws_foundation_stack_name=AnsibleAutomationPlatform aws_region=us-east-1 aws_ssm_bucket_name=aap-ssm-bucket aws_backup_taken=True seller_name=redhatinc'
  3. The upgrade can take some time to complete, but can take longer depending on the number of extension nodes on the system. A successful upgrade is marked by the log below.

    TASK [redhat.ansible_on_clouds.standalone_aws_upgrade : [upgrade] [LOG] upgrade version] ***
    ok: [localhost] => {
        "msg": "Successfully upgraded from '2.3.20230221-00' -> '2.4.20230630-00'."
    }
  4. Your Ansible Automation Platform from AWS Marketplace deployment is now upgraded to a newer version and you can log in to Red Hat Ansible Automation Platform automation controller and automation hub using your deployment credentials.

10.3. Restore from backup

If you would like to restore the earlier version of your Ansible Automation Platform environment to the state it was in before the upgrade, follow the Ansible on Clouds 2.3 restore instructions.