Chapter 13. Tutorial: Deploying the External DNS Operator on ROSA
Starting with Red Hat OpenShift Service on AWS 4.14, the Custom Domain Operator is deprecated. To manage Ingress in Red Hat OpenShift Service on AWS 4.14, use the Ingress Operator. The functionality is unchanged for Red Hat OpenShift Service on AWS 4.13 and earlier versions.
Configuring the Custom Domain Operator requires a wildcard CNAME DNS record in your Amazon Route 53 hosted zone. If you do not want to use a wildcard record, you can use the External DNS
Operator to create individual entries for routes.
Use this tutorial to deploy and configure the External DNS
Operator with a custom domain in Red Hat OpenShift Service on AWS (ROSA).
The External DNS
Operator does not support STS using IAM Roles for Service Accounts (IRSA) and uses long-lived Identity Access Management (IAM) credentials instead. This tutorial will be updated when the Operator supports STS.
13.1. Prerequisites
- A ROSA cluster
-
A user account with
dedicated-admin
privileges -
The OpenShift CLI (
oc
) -
The Amazon Web Services (AWS) CLI (
aws
) -
A unique domain, such as
*.apps.<company_name>.io
- An Amazon Route 53 public hosted zone for the above domain
13.2. Setting up your environment
Configure the following environment variables, replacing
CLUSTER_NAME
with the name of your cluster:$ export DOMAIN=apps.<company_name>.io 1 $ export AWS_PAGER="" $ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}$//') $ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}") $ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) $ export SCRATCH="/tmp/${CLUSTER_NAME}/external-dns" $ mkdir -p ${SCRATCH}
- 1
- The custom domain.
Ensure all fields output correctly before moving to the next section:
$ echo "Cluster: ${CLUSTER_NAME}, Region: ${REGION}, AWS Account ID: ${AWS_ACCOUNT_ID}"
13.3. Setting up your custom domain
ROSA manages secondary Ingress Controllers using the Custom Domain
Operator. Use the following procedure to deploy a secondary Ingress Controller using a custom domain.
Prerequisites
-
A unique domain, such as
*.apps.<company_name>.io
-
A custom SAN or wildcard certificate, such as
CN=*.apps.<company_name>.io
Procedure
Create a new project:
$ oc new-project external-dns-operator
Create a new TLS secret from a private key and a public certificate, where
fullchain.pem
is your full wildcard certificate chain (including any intermediaries) andprivkey.pem
is your wildcard certificate’s private key:$ oc -n external-dns-operator create secret tls external-dns-tls --cert=fullchain.pem --key=privkey.pem
Create a new
CustomDomain
custom resource (CR):Example
external-dns-custom-domain.yaml
apiVersion: managed.openshift.io/v1alpha1 kind: CustomDomain metadata: name: external-dns spec: domain: apps.<company_name>.io 1 scope: External loadBalancerType: NLB certificate: name: external-dns-tls namespace: external-dns-operator
- 1
- The custom domain.
Apply the CR:
$ oc apply -f external-dns-custom-domain.yaml
Verify that your custom domain Ingress Controller has been deployed and has a
Ready
status:$ oc get customdomains
Example output
NAME ENDPOINT DOMAIN STATUS external-dns xxrywp.<company_name>.cluster-01.opln.s1.openshiftapps.com *.apps.<company_name>.io Ready
13.4. Preparing your AWS account
Retrieve the Amazon Route 53 public hosted zone ID:
$ export ZONE_ID=$(aws route53 list-hosted-zones-by-name --output json \ --dns-name "${DOMAIN}." --query 'HostedZones[0]'.Id --out text | sed 's/\/hostedzone\///')
Create an AWS IAM Policy document that allows the
External DNS
Operator to update only the custom domain public hosted zone:$ cat << EOF > "${SCRATCH}/external-dns-policy.json" { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets" ], "Resource": [ "arn:aws:route53:::hostedzone/${ZONE_ID}" ] }, { "Effect": "Allow", "Action": [ "route53:ListHostedZones", "route53:ListResourceRecordSets" ], "Resource": [ "*" ] } ] } EOF
Create an AWS IAM policy:
$ export POLICY_ARN=$(aws iam create-policy --policy-name "${CLUSTER_NAME}-AllowExternalDNSUpdates" \ --policy-document file://${SCRATCH}/external-dns-policy.json \ --query 'Policy.Arn' --output text)
Create an AWS IAM user:
$ aws iam create-user --user-name "${CLUSTER_NAME}-external-dns-operator"
Attach the policy:
$ aws iam attach-user-policy --user-name "${CLUSTER_NAME}-external-dns-operator" --policy-arn $POLICY_ARN
NoteThis will be changed to STS using IRSA in the future.
Create AWS keys for the IAM user:
$ SECRET_ACCESS_KEY=$(aws iam create-access-key --user-name "${CLUSTER_NAME}-external-dns-operator")
Create static credentials:
$ cat << EOF > "${SCRATCH}/credentials" [default] aws_access_key_id = $(echo $SECRET_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId') aws_secret_access_key = $(echo $SECRET_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey') EOF
13.5. Installing the External DNS Operator
Install the
External DNS
Operator from OperatorHub:$ cat << EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: external-dns-group namespace: external-dns-operator spec: targetNamespaces: - external-dns-operator --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: external-dns-operator namespace: external-dns-operator spec: channel: stable-v1.1 installPlanApproval: Automatic name: external-dns-operator source: redhat-operators sourceNamespace: openshift-marketplace EOF
Wait until the
External DNS
Operator is running:$ oc rollout status deploy external-dns-operator --timeout=300s
Create a secret from the AWS IAM user credentials:
$ oc -n external-dns-operator create secret generic external-dns \ --from-file "${SCRATCH}/credentials"
Deploy the
ExternalDNS
controller:$ cat << EOF | oc apply -f - apiVersion: externaldns.olm.openshift.io/v1beta1 kind: ExternalDNS metadata: name: ${DOMAIN} spec: domains: - filterType: Include matchType: Exact name: ${DOMAIN} provider: aws: credentials: name: external-dns type: AWS source: openshiftRouteOptions: routerName: external-dns type: OpenShiftRoute zones: - ${ZONE_ID} EOF
Wait until the controller is running:
$ oc rollout status deploy external-dns-${DOMAIN} --timeout=300s
13.6. Deploying a sample application
Now that the ExternalDNS
controller is running, you can deploy a sample application to confirm that the custom domain is configured and trusted when you expose a new route.
Create a new project for your sample application:
$ oc new-project hello-world
Deploy a hello world application:
$ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift
Create a route for the application specifying your custom domain name:
$ oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls \ --hostname hello-openshift.${DOMAIN}
Check if the DNS record was created automatically by ExternalDNS:
NoteIt can take a few minutes for the record to appear in Amazon Route 53.
$ aws route53 list-resource-record-sets --hosted-zone-id ${ZONE_ID} \ --query "ResourceRecordSets[?Type == 'CNAME']" | grep hello-openshift
Optional: You can also view the TXT records that indicate they were created by ExternalDNS:
$ aws route53 list-resource-record-sets --hosted-zone-id ${ZONE_ID} \ --query "ResourceRecordSets[?Type == 'TXT']" | grep ${DOMAIN}
Navigate to your custom console domain in the browser where you see the OpenShift login:
$ echo console.${DOMAIN}