Chapter 23. AWS Load Balancer Operator

Procedure

1. You can deploy the AWS Load Balancer Operator on demand from the OperatorHub, by creating a Subscription object:

   $ oc -n aws-load-balancer-operator get sub aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}'

   Example output

   install-zlfbt

2. Check the status of an install plan. The status of an install plan must be Complete:

   $ oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}'

   Example output

   Complete

3. Use the oc get command to view the Deployment status:

   $ oc get -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager

   Example output

   NAME                                           READY     UP-TO-DATE   AVAILABLE   AGE
   aws-load-balancer-operator-controller-manager  1/1       1            1           23h

Procedure

1. Navigate to Operators → OperatorHub in the OpenShift Container Platform web console.
2. Select the AWS Load Balancer Operator. You can use the Filter by keyword text box or use the filter list to search for the AWS Load Balancer Operator from the list of Operators.
3. Select the aws-load-balancer-operator namespace.
4. Follow the instructions to prepare the Operator for installation.
5. On the AWS Load Balancer Operator page, click Install.

6. On the Install Operator page, select the following options:

   1. Update the channel as stable-v1.
   2. Installation mode as A specific namespace on the cluster.
   3. Installed Namespace as aws-load-balancer-operator. If the aws-load-balancer-operator namespace does not exist, it gets created during the Operator installation.
   4. Select Update approval as Automatic or Manual. By default, the Update approval is set to Automatic. If you select automatic updates, the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention. If you select manual updates, the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator updated to the new version.
   5. Click Install.

Procedure

1. Create the aws-load-balancer-operator namespace by running the following command:

   $ oc create namespace aws-load-balancer-operator

2. Download the CredentialsRequest custom resource (CR) of the AWS Load Balancer Operator, and create a directory to store it by running the following command:

   $ curl --create-dirs -o <path-to-credrequests-dir>/cr.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/operator-credentials-request.yaml

3. Use the ccoctl tool to process CredentialsRequest objects of the AWS Load Balancer Operator, by running the following command:

   $ ccoctl aws create-iam-roles \
       --name <name> --region=<aws_region> \
       --credentials-requests-dir=<path-to-credrequests-dir> \
       --identity-provider-arn <oidc-arn>

4. Apply the secrets generated in the manifests directory of your cluster by running the following command:

   $ ls manifests/*-credentials.yaml | xargs -I{} oc apply -f {}

5. Verify that the credentials secret of the AWS Load Balancer Operator is created by running the following command:

   $ oc -n aws-load-balancer-operator get secret aws-load-balancer-operator --template='{{index .data "credentials"}}' | base64 -d

   Example output

   [default]
   sts_regional_endpoints = regional
   role_arn = arn:aws:iam::999999999999:role/aws-load-balancer-operator-aws-load-balancer-operator
   web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token

Procedure

1. The AWS Load Balancer Operator creates the CredentialsRequest object in the openshift-cloud-credential-operator namespace for each AWSLoadBalancerController custom resource (CR). You can extract and save the created CredentialsRequest object in a directory by running the following command:

   $ oc get credentialsrequest -n openshift-cloud-credential-operator  \
       aws-load-balancer-controller-<cr-name> -o yaml > <path-to-credrequests-dir>/cr.yaml 1

   1

   The aws-load-balancer-controller-<cr-name> parameter specifies the credential request name created by the AWS Load Balancer Operator. The cr-name specifies the name of the AWS Load Balancer Controller instance.

2. Use the ccoctl tool to process all CredentialsRequest objects in the credrequests directory by running the following command:

   $ ccoctl aws create-iam-roles \
       --name <name> --region=<aws_region> \
       --credentials-requests-dir=<path-to-credrequests-dir> \
       --identity-provider-arn <oidc-arn>

3. Apply the secrets generated in manifests directory to your cluster, by running the following command:

   $ ls manifests/*-credentials.yaml | xargs -I{} oc apply -f {}

4. Verify that the aws-load-balancer-controller pod is created:

   $ oc -n aws-load-balancer-operator get pods
   NAME                                                            READY   STATUS    RESTARTS   AGE
   aws-load-balancer-controller-cluster-9b766d6-gg82c              1/1     Running   0          137m
   aws-load-balancer-operator-controller-manager-b55ff68cc-85jzg   2/2     Running   0          3h26m

Procedure

1. Download the CredentialsRequest custom resource (CR) of the AWS Load Balancer Controller, and create a directory to store it by running the following command:

   $ curl --create-dirs -o <path-to-credrequests-dir>/cr.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/controller/controller-credentials-request.yaml

2. Use the ccoctl tool to process the CredentialsRequest object of the controller:

   $ ccoctl aws create-iam-roles \
           --name <name> --region=<aws_region> \
           --credentials-requests-dir=<path-to-credrequests-dir> \
           --identity-provider-arn <oidc-arn>

3. Apply the secrets to your cluster:

   $ ls manifests/*-credentials.yaml | xargs -I{} oc apply -f {}

4. Verify the credentials secret has been created for use by the controller:

   $ oc -n aws-load-balancer-operator get secret aws-load-balancer-controller-manual-cluster --template='{{index .data "credentials"}}' | base64 -d

   Example output

   [default]
       sts_regional_endpoints = regional
       role_arn = arn:aws:iam::999999999999:role/aws-load-balancer-operator-aws-load-balancer-controller
       web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token

5. Create the AWSLoadBalancerController resource YAML file, for example, sample-aws-lb-manual-creds.yaml, as follows:

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController 1
   metadata:
     name: cluster 2
   spec:
     credentials:
       name: <secret-name> 3

   1

   Defines the AWSLoadBalancerController resource.

   2

   Defines the AWS Load Balancer Controller instance name. This instance name gets added as a suffix to all related resources.

   3

   Specifies the secret name containing AWS credentials that the controller uses.

Procedure

1. Create an aws-load-balancer-controller resource YAML file, for example, sample-aws-lb.yaml, as follows:

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController 1
   metadata:
     name: cluster 2
   spec:
     subnetTagging: Auto 3
     additionalResourceTags: 4
     - key: example.org/security-scope
       value: staging
     ingressClass: cloud 5
     config:
       replicas: 2 6
     enabledAddons: 7
       - AWSWAFv2 8

   1

   Defines the aws-load-balancer-controller resource.

   2

   Defines the AWS Load Balancer Controller instance name. This instance name gets added as a suffix to all related resources.

   3

   Valid options are Auto and Manual. When the value is set to Auto, the Operator attempts to determine the subnets that belong to the cluster and tags them appropriately. The Operator cannot determine the role correctly if the internal subnet tags are not present on internal subnet. If you installed your cluster on user-provided infrastructure, you can manually tag the subnets with the appropriate role tags and set the subnet tagging policy to Manual.

   4

   Defines the tags used by the controller when it provisions AWS resources.

   5

   The default value for this field is alb. The Operator provisions an IngressClass resource with the same name if it does not exist.

   6

   Specifies the number of replicas of the controller.

   7

   Specifies add-ons for AWS load balancers, which get specified through annotations.

   8

   Enables the alb.ingress.kubernetes.io/wafv2-acl-arn annotation.

2. Create a aws-load-balancer-controller resource by running the following command:

   $ oc create -f sample-aws-lb.yaml

3. After the AWS Load Balancer Controller is running, create a deployment resource:

   apiVersion: apps/v1
   kind: Deployment 1
   metadata:
     name: <echoserver> 2
     namespace: echoserver
   spec:
     selector:
       matchLabels:
         app: echoserver
     replicas: 3 3
     template:
       metadata:
         labels:
           app: echoserver
       spec:
         containers:
           - image: openshift/origin-node
             command:
              - "/bin/socat"
             args:
               - TCP4-LISTEN:8080,reuseaddr,fork
               - EXEC:'/bin/bash -c \"printf \\\"HTTP/1.0 200 OK\r\n\r\n\\\"; sed -e \\\"/^\r/q\\\"\"'
             imagePullPolicy: Always
             name: echoserver
             ports:
               - containerPort: 8080

   1

   Defines the deployment resource.

   2

   Specifies the deployment name.

   3

   Specifies the number of replicas of the deployment.

4. Create a service resource:

   apiVersion: v1
   kind: Service 1
   metadata:
     name: <echoserver> 2
     namespace: echoserver
   spec:
     ports:
       - port: 80
         targetPort: 8080
         protocol: TCP
     type: NodePort
     selector:
       app: echoserver

   1

   Defines the service resource.

   2

   Specifies the name of the service.

5. Deploy an ALB-backed Ingress resource:

   apiVersion: networking.k8s.io/v1
   kind: Ingress 1
   metadata:
     name: <echoserver> 2
     namespace: echoserver
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/target-type: instance
   spec:
     ingressClassName: alb
     rules:
       - http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: <echoserver> 3
                   port:
                     number: 80

   1

   Defines the ingress resource.

   2

   Specifies the name of the ingress resource.

   3

   Specifies the name of the service resource.

Procedure

1. Create an IngressClassParams resource YAML file, for example, sample-single-lb-params.yaml, as follows:

   apiVersion: elbv2.k8s.aws/v1beta1 1
   kind: IngressClassParams
   metadata:
     name: single-lb-params 2
   spec:
     group:
       name: single-lb 3

   1

   Defines the API group and version of the IngressClassParams resource.

   2

   Specifies the name of the IngressClassParams resource.

   3

   Specifies the name of the IngressGroup. All Ingresses of this class belong to this IngressGroup.

2. Create an IngressClassParams resource by running the following command:

   $ oc create -f sample-single-lb-params.yaml

3. Create an IngressClass resource YAML file, for example, sample-single-lb-class.yaml, as follows:

   apiVersion: networking.k8s.io/v1 1
   kind: IngressClass
   metadata:
     name: single-lb 2
   spec:
     controller: ingress.k8s.aws/alb 3
     parameters:
       apiGroup: elbv2.k8s.aws 4
       kind: IngressClassParams 5
       name: single-lb-params 6

   1

   Defines the API group and version of the IngressClass resource.

   2

   Specifies the name of the IngressClass.

   3

   Defines the controller name. ingress.k8s.aws/alb denotes that all Ingresses of this class should be managed by the aws-load-balancer-controller.

   4

   Defines the API group of the IngressClassParams resource.

   5

   Defines the resource type of the IngressClassParams resource.

   6

   Defines the name of the IngressClassParams resource.

4. Create an IngressClass resource by running the following command:

   $ oc create -f sample-single-lb-class.yaml

5. Create an AWSLoadBalancerController resource YAML file, for example, sample-single-lb.yaml, as follows:

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController
   metadata:
     name: cluster
   spec:
     subnetTagging: Auto
     ingressClass: single-lb 1

   1

   Defines the name of the IngressClass resource.

6. Create an AWSLoadBalancerController resource by running the following command:

   $ oc create -f sample-single-lb.yaml

7. Create an Ingress resource YAML file, for example, sample-multiple-ingress.yaml, as follows:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: example-1 1
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing 2
       alb.ingress.kubernetes.io/group.order: "1" 3
       alb.ingress.kubernetes.io/target-type: instance 4
   spec:
     ingressClassName: single-lb 5
     rules:
     - host: example.com 6
       http:
           paths:
           - path: /blog 7
             pathType: Prefix
             backend:
               service:
                 name: example-1 8
                 port:
                   number: 80 9
   ---
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: example-2
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/group.order: "2"
       alb.ingress.kubernetes.io/target-type: instance
   spec:
     ingressClassName: single-lb
     rules:
     - host: example.com
       http:
           paths:
           - path: /store
             pathType: Prefix
             backend:
               service:
                 name: example-2
                 port:
                   number: 80
   ---
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: example-3
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/group.order: "3"
       alb.ingress.kubernetes.io/target-type: instance
   spec:
     ingressClassName: single-lb
     rules:
     - host: example.com
       http:
           paths:
           - path: /
             pathType: Prefix
             backend:
               service:
                 name: example-3
                 port:
                   number: 80

   1

   Specifies the name of an ingress.

   2

   Indicates the load balancer to provision in the public subnet and makes it accessible over the internet.

   3

   Specifies the order in which the rules from the Ingresses are matched when the request is received at the load balancer.

   4

   Indicates the load balancer will target OpenShift nodes to reach the service.

   5

   Specifies the Ingress Class that belongs to this ingress.

   6

   Defines the name of a domain used for request routing.

   7

   Defines the path that must route to the service.

   8

   Defines the name of the service that serves the endpoint configured in the ingress.

   9

   Defines the port on the service that serves the endpoint.

8. Create the Ingress resources by running the following command:

   $ oc create -f sample-multiple-ingress.yaml

Procedure

1. Install the Operator and create an instance of the aws-load-balancer-controller resource:

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController
   metadata:
     name: cluster
   spec:
     subnetTagging: Auto
     ingressClass: tls-termination 1

   1

   Defines the name of an ingressClass resource reconciled by the AWS Load Balancer Controller. This ingressClass resource gets created if it is not present. You can add additional ingressClass values. The controller reconciles the ingressClass values if the spec.controller is set to ingress.k8s.aws/alb.

2. Create an Ingress resource:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: <example> 1
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing 2
       alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxxx 3
   spec:
     ingressClassName: tls-termination 4
     rules:
     - host: <example.com> 5
       http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: <example-service> 6
                   port:
                     number: 80

   1

   Specifies the name of an ingress.

   2

   The controller provisions the load balancer for this Ingress resource in a public subnet so that the load balancer is reachable over the internet.

   3

   The Amazon Resource Name of the certificate that you attach to the load balancer.

   4

   Defines the ingress class name.

   5

   Defines the domain for traffic routing.

   6

   Defines the service for traffic routing.

1. Create the config map to contain the certificate authority (CA) bundle in the aws-load-balancer-operator namespace and inject a CA bundle that is trusted by OpenShift Container Platform into a config map by running the following command:

   $ oc -n aws-load-balancer-operator create configmap trusted-ca

2. To inject the trusted CA bundle into the config map, add the config.openshift.io/inject-trusted-cabundle=true label to the config map by running the following command:

   $ oc -n aws-load-balancer-operator label cm trusted-ca config.openshift.io/inject-trusted-cabundle=true

3. Update the subscription of the AWS Load Balancer Operator to access the config map in the deployment of the AWS Load Balancer Operator by running the following command:

   $ oc -n aws-load-balancer-operator patch subscription aws-load-balancer-operator --type='merge' -p '{"spec":{"config":{"env":[{"name":"TRUSTED_CA_CONFIGMAP_NAME","value":"trusted-ca"}],"volumes":[{"name":"trusted-ca","configMap":{"name":"trusted-ca"}}],"volumeMounts":[{"name":"trusted-ca","mountPath":"/etc/pki/tls/certs/albo-tls-ca-bundle.crt","subPath":"ca-bundle.crt"}]}}}'

4. After the deployment of the AWS Load Balancer Operator is completed, verify that the CA bundle is added to the aws-load-balancer-operator-controller-manager deployment by running the following command:

   $ oc -n aws-load-balancer-operator exec deploy/aws-load-balancer-operator-controller-manager -c manager -- bash -c "ls -l /etc/pki/tls/certs/albo-tls-ca-bundle.crt; printenv TRUSTED_CA_CONFIGMAP_NAME"

   Example output

   -rw-r--r--. 1 root 1000690000 5875 Jan 11 12:25 /etc/pki/tls/certs/albo-tls-ca-bundle.crt
   trusted-ca

5. Optional: Restart deployment of the AWS Load Balancer Operator every time the config map changes by running the following command:

   $ oc -n aws-load-balancer-operator rollout restart deployment/aws-load-balancer-operator-controller-manager