Chapter 5. GitOps

1. Application repository with the source code
2. Environment configuration repository that defines the desired state of the application

Procedure

1. Open the Administrator perspective of the web console and navigate to Operators → OperatorHub in the menu on the left.

2. Search for OpenShift GitOps, click the Red Hat OpenShift GitOps tile, and then click Install.

   Red Hat OpenShift GitOps will be installed in all namespaces of the cluster.

Procedure

1. In the Administrator perspective of the web console, navigate to Operators → Installed Operators to verify that the Red Hat OpenShift GitOps Operator is installed.
2. Navigate to the menu → OpenShift GitOps → Cluster Argo CD. The login page of the Argo CD UI is displayed in a new window.

3. Obtain the password for the Argo CD instance:

   1. In the left panel of the console, use the perspective switcher to switch to the Developer perspective.
   2. Use the Project drop-down list and select the openshift-gitops project.
   3. Use the left navigation panel to navigate to the Secrets page.
   4. Select the openshift-gitops-cluster instance to display the password.
   5. Copy the password.

1. Use this password and admin as the username to log in to the Argo CD UI in the new window.

Procedure

1. In the Terminal type the following command:

Procedure

1. From the Operators → OperatorHub page, use the Filter by keyword box to search for Red Hat OpenShift GitOps Operator tile.
2. Click the Red Hat OpenShift GitOps Operator tile. The Operator tile indicates it is installed.
3. In the Red Hat OpenShift GitOps Operator descriptor page, click Uninstall.

Procedure

1. In the Administrator perspective of the web console, navigate to Operators → Installed Operators → Red Hat OpenShift GitOps → Subscription.
2. Click the Actions drop-down menu then click Edit Subscription.

3. On the openshift-gitops-operator Subscription details page, under the YAML tab, edit the Subscription YAML file by adding the namespace of the Argo CD instance to the ARGOCD_CLUSTER_CONFIG_NAMESPACES environment variable in the spec section:

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: openshift-gitops-operator
     namespace: openshift-operators
   ...
   spec:
     config:
       env:
       - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES
         value: openshift-gitops, <list of namespaces of cluster-scoped Argo CD instances>
   ...

4. To verify that the Argo instance is configured with a cluster role to manage cluster-scoped resources, perform the following steps:

   1. Navigate to User Management → Roles and from the Filter drop-down menu select Cluster-wide Roles.

   2. Search for the argocd-application-controller by using the Search by name field.

      The Roles page displays the created cluster role.

      Tip

      Alternatively, in the OpenShift CLI, run the following command:

      oc auth can-i create oauth -n openshift-gitops --as system:serviceaccount:openshift-gitops:openshift-gitops-argocd-application-controller

      The output yes verifies that the Argo instance is configured with a cluster role to manage cluster-scoped resources. Else, check your configurations and take necessary steps as required.

Procedure

1. Label the existing nodes:

   $ oc label node <node-name> node-role.kubernetes.io/infra=""

2. Optional: If required, you can also apply taints and isolate the workloads on infrastructure nodes and prevent other workloads from scheduling on these nodes:

   $ oc adm taint nodes -l node-role.kubernetes.io/infra \
   infra=reserved:NoSchedule infra=reserved:NoExecute

3. Add the runOnInfra toggle in the GitOpsService custom resource:

   apiVersion: pipelines.openshift.io/v1alpha1
   kind: GitopsService
   metadata:
     name: cluster
   spec:
     runOnInfra: true

4. Optional: If taints have been added to the nodes, then add tolerations to the GitOpsService custom resource, for example:

     spec:
       runOnInfra: true
       tolerations:
       - effect: NoSchedule
         key: infra
         value: reserved
       - effect: NoExecute
         key: infra
         value: reserved

5. Verify that the workloads in the openshift-gitops namespace are now scheduled on the infrastructure nodes by viewing Pods → Pod details for any pod in the console UI.

Procedure

1. In the Argo CD dashboard, click NEW APP to add a new Argo CD application.

2. For this workflow, create a cluster-configs application with the following configurations:

   Application Name

   cluster-configs

   Project

   default

   Sync Policy

   Manual

   Repository URL

   https://github.com/redhat-developer/openshift-gitops-getting-started

   Revision

   HEAD

   Path

   cluster

   Destination

   https://kubernetes.default.svc

   Namespace

   spring-petclinic

   Directory Recurse

   checked

3. Click CREATE to create your application.
4. Open the Administrator perspective of the web console and navigate to Administration → Namespaces in the menu on the left.
5. Search for and select the namespace, then enter argocd.argoproj.io/managed-by=openshift-gitops in the Label field so that the Argo CD instance in the openshift-gitops namespace can manage your namespace.

Procedure

1. Download the sample application:

   $ git clone git@github.com:redhat-developer/openshift-gitops-getting-started.git

2. Create the application:

   $ oc create -f openshift-gitops-getting-started/argo/cluster.yaml

3. Run the oc get command to review the created application:

   $ oc get application -n openshift-gitops

4. Add a label to the namespace your application is deployed in so that the Argo CD instance in the openshift-gitops namespace can manage it:

   $ oc label namespace spring-petclinic argocd.argoproj.io/managed-by=openshift-gitops

Procedure

1. In the Argo CD dashboard, notice that the cluster-configs Argo CD application has the statuses Missing and OutOfSync. Because the application was configured with a manual sync policy, Argo CD does not sync it automatically.
2. Click SYNC on the cluster-configs tile, review the changes, and then click SYNCHRONIZE. Argo CD will detect any changes in the Git repository automatically. If the configurations are changed, Argo CD will change the status of the cluster-configs to OutOfSync. You can modify the synchronization policy for Argo CD to automatically apply changes from your Git repository to the cluster.
3. Notice that the cluster-configs Argo CD application now has the statuses Healthy and Synced. Click the cluster-configs tile to check the details of the synchronized resources and their status on the cluster.
4. Navigate to the OpenShift Container Platform web console and click to verify that a link to the Red Hat Developer Blog - Kubernetes is now present there.

5. Navigate to the Project page and search for the spring-petclinic namespace to verify that it has been added to the cluster.

   Your cluster configurations have been successfully synchronized to the cluster.

Procedure

1. Log in to the OpenShift Container Platform web console as an admin.

2. In the web console, select User Management → Roles → Create Role. Use the following ClusterRole YAML template to add rules to specify the additional permissions.

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: secrets-cluster-role
   rules:
   - apiGroups: [""]
     resources: ["secrets"]
     verbs: ["*"]

3. Click Create to add the cluster role.
4. Now create the cluster role binding. In the web console, select User Management → Role Bindings → Create Binding.
5. Select All Projects from the Project drop-down.
6. Click Create binding.
7. Select Binding type as Cluster-wide role binding (ClusterRoleBinding).
8. Enter a unique value for the RoleBinding name.
9. Select the newly created cluster role or an existing cluster role from the drop down list.

10. Select the Subject as ServiceAccount and the provide the Subject namespace and name.

    1. Subject namespace: openshift-gitops
    2. Subject name: openshift-gitops-argocd-application-controller

11. Click Create. The YAML file for the ClusterRoleBinding object is as follows:

    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: cluster-role-binding
    subjects:
      - kind: ServiceAccount
        name: openshift-gitops-argocd-application-controller
        namespace: openshift-gitops
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: admin

Procedure

1. In the Argo CD dashboard, click NEW APP to add a new Argo CD application.

2. For this workflow, create a cluster-configs application with the following configurations:

   Application Name

   cluster-configs

   Project

   default

   Sync Policy

   Manual

   Repository URL

   https://github.com/redhat-developer/openshift-gitops-getting-started

   Revision

   HEAD

   Path

   cluster

   Destination

   https://kubernetes.default.svc

   Namespace

   spring-petclinic

   Directory Recurse

   checked

3. For this workflow, create a spring-petclinic application with the following configurations:

   Application Name

   spring-petclinic

   Project

   default

   Sync Policy

   Automatic

   Repository URL

   https://github.com/redhat-developer/openshift-gitops-getting-started

   Revision

   HEAD

   Path

   app

   Destination

   https://kubernetes.default.svc

   Namespace

   spring-petclinic

4. Click CREATE to create your application.
5. Open the Administrator perspective of the web console and navigate to Administration → Namespaces in the menu on the left.
6. Search for and select the namespace, then enter argocd.argoproj.io/managed-by=openshift-gitops in the Label field so that the Argo CD instance in the openshift-gitops namespace can manage your namespace.

Procedure

1. Download the sample application:

   $ git clone git@github.com:redhat-developer/openshift-gitops-getting-started.git

2. Create the application:

   $ oc create -f openshift-gitops-getting-started/argo/app.yaml

   $ oc create -f openshift-gitops-getting-started/argo/cluster.yaml

3. Run the oc get command to review the created application:

   $ oc get application -n openshift-gitops

4. Add a label to the namespace your application is deployed in so that the Argo CD instance in the openshift-gitops namespace can manage it:

   $ oc label namespace spring-petclinic argocd.argoproj.io/managed-by=openshift-gitops

   $ oc label namespace spring-petclinic argocd.argoproj.io/managed-by=openshift-gitops

Procedure

1. In the Argo CD dashboard, verify that your application has the Synced status.
2. Click the app-spring-petclinic tile in the Argo CD dashboard to view the application resources that are deployed to the cluster.
3. In the OpenShift Container Platform web console, navigate to the Developer perspective.

4. Modify the Spring PetClinic deployment and commit the changes to the app/ directory of the Git repository. Argo CD will automatically deploy the changes to the cluster.

   1. Fork the OpenShift GitOps getting started repository.
   2. In the deployment.yaml file, change the failureThreshold value to 5.

   3. In the deployment cluster, run the following command to verify the changed value of the failureThreshold field:

      $ oc edit deployment spring-petclinic -n spring-petclinic

5. Test the self-healing behavior by modifying the deployment on the cluster and scaling it up to two pods while watching the application in the OpenShift Container Platform web console.

   1. Run the following command to modify the deployment:

      $ oc scale deployment spring-petclinic --replicas 2  -n spring-petclinic

   2. In the OpenShift Container Platform web console, notice that the deployment scales up to two pods and immediately scales down again to one pod. Argo CD detected a difference from the Git repository and auto-healed the application on the OpenShift Container Platform cluster.
6. In the Argo CD dashboard, click the app-spring-petclinic tile → APP DETAILS → EVENTS. The EVENTS tab displays the following events: Argo CD detecting out of sync deployment resources on the cluster and then resyncing the Git repository to correct it.

1. Delete the Dex configuration by removing the following section from the Argo CD Custom Resource (CR), and save the CR:

   dex:
       openShiftOAuth: true
       resources:
         limits:
           cpu:
           memory:
         requests:
           cpu:
           memory:

2. Configure Keycloak by editing the Argo CD CR, and updating the value for the provider parameter as keycloak. For example:

   apiVersion: argoproj.io/v1alpha1
   kind: ArgoCD
   metadata:
     name: example-argocd
     labels:
       example: basic
   spec:
     sso:
       provider: keycloak
     server:
       route:
        enabled: true

Procedure

1. Get the Keycloak route URL for login:

   $ oc -n argocd get route keycloak
   
   NAME        HOST/PORT                                                        PATH   SERVICES   PORT    TERMINATION   WILDCARD
   keycloak    keycloak-default.apps.ci-ln-******.origin-ci-int-aws.dev.**.com         keycloak   <all>    reencrypt     None

2. Get the Keycloak pod name that stores the user name and password as environment variables:

   $ oc -n argocd get pods
   
   NAME                      READY   STATUS           RESTARTS   AGE
   keycloak-1-2sjcl           1/1    Running            0        45m

   1. Get the Keycloak user name:

      $ oc -n argocd exec keycloak-1-2sjcl -- "env" | grep SSO_ADMIN_USERNAME
      
      SSO_ADMIN_USERNAME=Cqid54Ih

   2. Get the Keycloak password:

      $ oc -n argocd exec keycloak-1-2sjcl -- "env" | grep SSO_ADMIN_PASSWORD
      
      SSO_ADMIN_PASSWORD=GVXxHifH

3. On the login page, click LOG IN VIA KEYCLOAK.

   Note

   You only see the option LOGIN VIA KEYCLOAK after the Keycloak instance is ready.

4. Click Login with OpenShift.

   Note

   Login using kubeadmin is not supported.

5. Enter the OpenShift credentials to log in.

6. Optional: By default, any user logged in to Argo CD has read-only access. You can manage the user level access by updating the argocd-rbac-cm config map:

   policy.csv:
   <name>, <email>, role:admin

1. Log in to the Keycloak pod:

   $ oc exec -it dc/keycloak -n argocd -- /bin/bash

2. Launch the JBoss CLI tool to set up the proxy mappings:

   /opt/eap/bin/jboss-cli.sh

3. In the JBoss CLI tool, run the following command to start an embedded standalone server:

   embed-server --server-config=standalone-openshift.xml

4. Set up proxy mappings for the OpenShift OAuth server host:

   /subsystem=keycloak-server/spi=connectionsHttpClient/provider=default:write-attribute(name=properties.proxy-mappings,value=["<oauth-server-hostname>;http://<proxy-server-host>:<proxy-server-port>"])

5. Stop the embedded server:

   quit

6. Reload the JBoss CLI tool to apply the proxy mappings:

   /opt/eap/bin/jboss-cli.sh --connect --command=:reload

Procedure

1. Label existing nodes as infrastructure by running the following command:

   $ oc label node <node-name> node-role.kubernetes.io/infra=

2. Edit the GitOpsService Custom Resource (CR) to add the infrastructure node selector:

   $ oc edit gitopsservice -n openshift-gitops

3. In the GitOpsService CR file, add runOnInfra field to the spec section and set it to true. This field moves the workloads in openshift-gitops namespace to the infrastructure nodes:

   apiVersion: pipelines.openshift.io/v1alpha1
   kind: GitopsService
   metadata:
     name: cluster
   spec:
     runOnInfra: true

4. Optional: Apply taints and isolate the workloads on infrastructure nodes and prevent other workloads from scheduling on these nodes.

   $ oc adm taint nodes -l node-role.kubernetes.io/infra
   infra=reserved:NoSchedule infra=reserved:NoExecute

5. Optional: If you apply taints to the nodes, you can add tolerations in the GitOpsService CR:

   spec:
     runOnInfra: true
     tolerations:
     - effect: NoSchedule
       key: infra
       value: reserved
     - effect: NoExecute
       key: infra
       value: reserved