Chapter 2. Pipelines

2.1. Red Hat OpenShift Pipelines release notes

Red Hat OpenShift Pipelines is a cloud-native CI/CD experience based on the Tekton project which provides:

  • Standard Kubernetes-native pipeline definitions (CRDs).
  • Serverless pipelines with no CI server management overhead.
  • Extensibility to build images using any Kubernetes tool, such as S2I, Buildah, JIB, and Kaniko.
  • Portability across any Kubernetes distribution.
  • Powerful CLI for interacting with pipelines.
  • Integrated user experience with the Developer perspective of the OpenShift Container Platform web console.

For an overview of Red Hat OpenShift Pipelines, see Understanding OpenShift Pipelines.

2.1.1. Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see Red Hat CTO Chris Wright’s message.

2.1.2. Release notes for Red Hat OpenShift Pipelines General Availability 1.4

Red Hat OpenShift Pipelines General Availability (GA) 1.4 is now available on OpenShift Container Platform 4.7.

Note

In addition to the stable and preview Operator channels, the Red Hat OpenShift Pipelines Operator 1.4.0 comes with the ocp-4.6, ocp-4.5, and ocp-4.4 deprecated channels. These deprecated channels and support for them will be removed in the following release of Red Hat OpenShift Pipelines.

2.1.2.1. Compatibility and support matrix

Some features in this release are currently in Technology Preview. These experimental features are not intended for production use.

Technology Preview Features Support Scope

In the table below, features are marked with the following statuses:

  • TP: Technology Preview
  • GA: General Availability

Note the following scope of support on the Red Hat Customer Portal for these features:

Table 2.1. Compatibility and support matrix

FeatureVersionSupport Status

Pipelines

0.22

GA

CLI

0.17

GA

Catalog

0.22

GA

Triggers

0.12

TP

Pipeline resources

-

TP

For questions and feedback, you can send an email to the product team at pipelines-interest@redhat.com.

2.1.2.2. New features

In addition to the fixes and stability improvements, the following sections highlight what is new in Red Hat OpenShift Pipelines 1.4.

  • The custom tasks have the following enhancements:

    • Pipeline results can now refer to results produced by custom tasks.
    • Custom tasks can now use workspaces, service accounts, and pod templates to build more complex custom tasks.
  • The finally task has the following enhancements:

    • The when expressions are supported in finally tasks, which provides efficient guarded execution and improved reusability of tasks.
    • A finally task can be configured to consume the results of any task within the same pipeline.

      Note

      Support for when expressions and finally tasks are unavailable in the OpenShift Container Platform 4.7 web console.

  • Support for multiple secrets of the type dockercfg or dockerconfigjson is added for authentication at runtime.
  • Functionality to support sparse-checkout with the git-clone task is added. This enables you to clone only a subset of the repository as your local copy, and helps you to restrict the size of the cloned repositories.
  • You can create pipeline runs in a pending state without actually starting them. In clusters that are under heavy load, this allows Operators to have control over the start time of the pipeline runs.
  • Ensure that you set the SYSTEM_NAMESPACE environment variable manually for the controller; this was previously set by default.
  • A non-root user is now added to the build-base image of pipelines so that git-init can clone repositories as a non-root user.
  • Support to validate dependencies between resolved resources before a pipeline run starts is added. All result variables in the pipeline must be valid, and optional workspaces from a pipeline can only be passed to tasks expecting it for the pipeline to start running.
  • The controller and webhook runs as a non-root group, and their superfluous capabilities have been removed to make them more secure.
  • You can use the tkn pr logs command to see the log streams for retried task runs.
  • You can use the --clustertask option in the tkn tr delete command to delete all the task runs associated with a particular cluster task.
  • Support for using Knative service with the EventListener resource is added by introducing a new customResource field.
  • An error message is displayed when an event payload does not use the JSON format.
  • The source control interceptors such as GitLab, BitBucket, and GitHub, now use the new InterceptorRequest or InterceptorResponse type interface.
  • A new CEL function marshalJSON is implemented so that you can encode a JSON object or an array to a string.
  • An HTTP handler for serving the CEL and the source control core interceptors is added. It packages four core interceptors into a single HTTP server that is deployed in the tekton-pipelines namespace. The EventListener object forwards events over the HTTP server to the interceptor. Each interceptor is available at a different path. For example, the CEL interceptor is available on the /cel path.
  • The pipelines-scc Security Context Constraint (SCC) is used with the default pipeline service account for pipelines. This new service account is similar to anyuid, but with a minor difference as defined in the YAML for SCC of OpenShift Container Platform 4.7:

    fsGroup:
      type: MustRunAs

2.1.2.3. Deprecated features

  • The build-gcs sub-type in the pipeline resource storage, and the gcs-fetcher image, are not supported.
  • In the taskRun field of cluster tasks, the label tekton.dev/task is removed.
  • For webhooks, the value v1beta1 corresponding to the field admissionReviewVersions is removed.
  • The creds-init helper image for building and deploying is removed.
  • In the triggers spec and binding, the deprecated field template.name is removed in favor of template.ref. You should update all eventListener definitions to use the ref field.

    Note

    Upgrade from Pipelines 1.3.x and earlier versions to Pipelines 1.4.0 breaks event listeners because of the unavailability of the template.name field. For such cases, use Pipelines 1.4.1 to avail the restored template.name field.

  • For EventListener custom resources/objects, the fields PodTemplate and ServiceType are deprecated in favor of Resource.
  • The deprecated spec style embedded bindings is removed.
  • The spec field is removed from the triggerSpecBinding.
  • The event ID representation is changed from a five-character random string to a UUID.

2.1.2.4. Known issues

  • In the Developer perspective, the pipeline metrics and triggers features are available only on OpenShift Container Platform 4.7.6 or later versions.
  • On IBM Power Systems, IBM Z, and LinuxONE, the tkn hub command is not supported.
  • When you run Maven and Jib Maven cluster tasks on an IBM Power Systems (ppc64le), IBM Z, and LinuxONE (s390x) clusters, set the MAVEN_IMAGE parameter value to maven:3.6.3-adoptopenjdk-11.
  • Triggers throw error resulting from bad handling of the JSON format, if you have the following configuration in the trigger binding:

    params:
      - name: github_json
        value: $(body)

    To resolve the issue:

    • If you are using triggers v0.11.0 and above, use the marshalJSON CEL function, which takes a JSON object or array and returns the JSON encoding of that object or array as a string.
    • If you are using older triggers version, add the following annotation in the trigger template:

      annotations:
        triggers.tekton.dev/old-escape-quotes: "true"
  • When upgrading from Pipelines 1.3.x to 1.4.x, you must recreate the routes.

2.1.2.5. Fixed issues

  • Previously, the tekton.dev/task label was removed from the task runs of cluster tasks, and the tekton.dev/clusterTask label was introduced. The problems resulting from that change is resolved by fixing the clustertask describe and delete commands. In addition, the lastrun function for tasks is modified, to fix the issue of the tekton.dev/task label being applied to the task runs of both tasks and cluster tasks in older versions of pipelines.
  • When doing an interactive tkn pipeline start pipelinename, a PipelineResource is created interactively. The tkn p start command prints the resource status if the resource status is not nil.
  • Previously, the tekton.dev/task=name label was removed from the task runs created from cluster tasks. This fix modifies the tkn clustertask start command with the --last flag to check for the tekton.dev/task=name label in the created task runs.
  • When a task uses an inline task specification, the corresponding task run now gets embedded in the pipeline when you run the tkn pipeline describe command, and the task name is returned as embedded.
  • The tkn version command is fixed to display the version of the installed Tekton CLI tool, without a configured kubeConfiguration namespace or access to a cluster.
  • If an argument is unexpected or more than one arguments are used, the tkn completion command gives an error.
  • Previously, pipeline runs with the finally tasks nested in a pipeline specification would lose those finally tasks, when converted to the v1alpha1 version and restored back to the v1beta1 version. This error occurring during conversion is fixed to avoid potential data loss. Pipeline runs with the finally tasks nested in a pipeline specification is now serialized and stored on the alpha version, only to be deserialized later.
  • Previously, there was an error in the pod generation when a service account had the secrets field as {}. The task runs failed with CouldntGetTask because the GET request with an empty secret name returned an error, indicating that the resource name may not be empty. This issue is fixed by avoiding an empty secret name in the kubeclient GET request.
  • Pipelines with the v1beta1 API versions can now be requested along with the v1alpha1 version, without losing the finally tasks. Applying the returned v1alpha1 version will store the resource as v1beta1, with the finally section restored to its original state.
  • Previously, an unset selfLink field in the controller caused an error in the Kubernetes v1.20 clusters. As a temporary fix, the CloudEvent source field is set to a value that matches the current source URI, without the value of the auto-populated selfLink field.
  • Previously, a secret name with dots such as gcr.io led to a task run creation failure. This happened because of the secret name being used internally as part of a volume mount name. The volume mount name conforms to the RFC1123 DNS label and disallows dots as part of the name. This issue is fixed by replacing the dot with a dash that results in a readable name.
  • Context variables are now validated in the finally tasks.
  • Previously, when the task run reconciler was passed a task run that did not have a previous status update containing the name of the pod it created, the task run reconciler listed the pods associated with the task run. The task run reconciler used the labels of the task run, which were propagated to the pod, to find the pod. Changing these labels while the task run was running, caused the code to not find the existing pod. As a result, duplicate pods were created. This issue is fixed by changing the task run reconciler to only use the tekton.dev/taskRun Tekton-controlled label when finding the pod.
  • Previously, when a pipeline accepted an optional workspace and passed it to a pipeline task, the pipeline run reconciler stopped with an error if the workspace was not provided, even if a missing workspace binding is a valid state for an optional workspace. This issue is fixed by ensuring that the pipeline run reconciler does not fail to create a task run, even if an optional workspace is not provided.
  • The sorted order of step statuses matches the order of step containers.
  • Previously, the task run status was set to unknown when a pod encountered the CreateContainerConfigError reason, which meant that the task and the pipeline ran until the pod timed out. This issue is fixed by setting the task run status to false, so that the task is set as failed when the pod encounters the CreateContainerConfigError reason.
  • Previously, pipeline results were resolved on the first reconciliation, after a pipeline run was completed. This could fail the resolution resulting in the Succeeded condition of the pipeline run being overwritten. As a result, the final status information was lost, potentially confusing any services watching the pipeline run conditions. This issue is fixed by moving the resolution of pipeline results to the end of a reconciliation, when the pipeline run is put into a Succeeded or True condition.
  • Execution status variable is now validated. This avoids validating task results while validating context variables to access execution status.
  • Previously, a pipeline result that contained an invalid variable would be added to the pipeline run with the literal expression of the variable intact. Therefore, it was difficult to assess whether the results were populated correctly. This issue is fixed by filtering out the pipeline run results that reference failed task runs. Now, a pipeline result that contains an invalid variable will not be emitted by the pipeline run at all.
  • The tkn eventlistener describe command is fixed to avoid crashing without a template. It also displays the details about trigger references.
  • Upgrades from Pipelines 1.3.x and earlier versions to Pipelines 1.4.0 breaks event listeners because of the unavailability of template.name. In Pipelines 1.4.1, the template.name has been restored to avoid breaking event listeners in triggers.
  • In Pipelines 1.4.1, the ConsoleQuickStart custom resource has been updated to align with OpenShift Container Platform 4.7 capabilities and behavior.

2.1.3. Release notes for Red Hat OpenShift Pipelines Technology Preview 1.3

2.1.3.1. New features

Red Hat OpenShift Pipelines Technology Preview (TP) 1.3 is now available on OpenShift Container Platform 4.7. Red Hat OpenShift Pipelines TP 1.3 is updated to support:

  • Tekton Pipelines 0.19.0
  • Tekton tkn CLI 0.15.0
  • Tekton Triggers 0.10.2
  • cluster tasks based on Tekton Catalog 0.19.0
  • IBM Power Systems on OpenShift Container Platform 4.7
  • IBM Z and LinuxONE on OpenShift Container Platform 4.7

In addition to the fixes and stability improvements, the following sections highlight what is new in Red Hat OpenShift Pipelines 1.3.

2.1.3.1.1. Pipelines
  • Tasks that build images, such as S2I and Buildah tasks, now emit a URL of the image built that includes the image SHA.
  • Conditions in pipeline tasks that reference custom tasks are disallowed because the Condition custom resource definition (CRD) has been deprecated.
  • Variable expansion is now added in the Task CRD for the following fields: spec.steps[].imagePullPolicy and spec.sidecar[].imagePullPolicy.
  • You can disable the built-in credential mechanism in Tekton by setting the disable-creds-init feature-flag to true.
  • Resolved when expressions are now listed in the Skipped Tasks and the Task Runs sections in the Status field of the PipelineRun configuration.
  • The git init command can now clone recursive submodules.
  • A Task CR author can now specify a timeout for a step in the Task spec.
  • You can now base the entry point image on the distroless/static:nonroot image and give it a mode to copy itself to the destination, without relying on the cp command being present in the base image.
  • You can now use the configuration flag require-git-ssh-secret-known-hosts to disallow omitting known hosts in the Git SSH secret. When the flag value is set to true, you must include the known_host field in the Git SSH secret. The default value for the flag is false.
  • The concept of optional workspaces is now introduced. A task or pipeline might declare a workspace optional and conditionally change their behavior based on its presence. A task run or pipeline run might also omit that workspace, thereby modifying the task or pipeline behavior. The default task run workspaces are not added in place of an omitted optional workspace.
  • Credentials initialization in Tekton now detects an SSH credential that is used with a non-SSH URL, and vice versa in Git pipeline resources, and logs a warning in the step containers.
  • The task run controller emits a warning event if the affinity specified by the pod template is overwritten by the affinity assistant.
  • The task run reconciler now records metrics for cloud events that are emitted once a task run is completed. This includes retries.
2.1.3.1.2. Pipelines CLI
  • Support for --no-headers flag is now added to the following commands: tkn condition list,tkn triggerbinding list,tkn eventlistener list,tkn clustertask list, tkn clustertriggerbinding list.
  • When used together, the --last or --use options override the --prefix-name and --timeout options.
  • The tkn eventlistener logs command is now added to view the EventListener logs.
  • The tekton hub commands are now integrated into the tkn CLI.
  • The --nocolour option is now changed to --no-color.
  • The --all-namespaces flag is added to the following commands: tkn triggertemplate list, tkn condition list, tkn triggerbinding list, tkn eventlistener list.
2.1.3.1.3. Triggers
  • You can now specify your resource information in the EventListener template.
  • It is now mandatory for EventListener service accounts to have the list and watch verbs, in addition to the get verb for all the triggers resources. This enables you to use Listers to fetch data from EventListener, Trigger, TriggerBinding, TriggerTemplate, and ClusterTriggerBinding resources. You can use this feature to create a Sink object rather than specifying multiple informers, and directly make calls to the API server.
  • A new Interceptor interface is added to support immutable input event bodies. Interceptors can now add data or fields to a new extensions field, and cannot modify the input bodies making them immutable. The CEL interceptor uses this new Interceptor interface.
  • A namespaceSelector field is added to the EventListener resource. Use it to specify the namespaces from where the EventListener resource can fetch the Trigger object for processing events. To use the namespaceSelector field, the service account for the EventListener resource must have a cluster role.
  • The triggers EventListener resource now supports end-to-end secure connection to the eventlistener pod.
  • The escaping parameters behavior in the TriggerTemplates resource by replacing " with \" is now removed.
  • A new resources field, supporting Kubernetes resources, is introduced as part of the EventListener spec.
  • A new functionality for the CEL interceptor, with support for upper and lower-casing of ASCII strings, is added.
  • You can embed TriggerBinding resources by using the name and value fields in a trigger, or an event listener.
  • The PodSecurityPolicy configuration is updated to run in restricted environments. It ensures that containers must run as non-root. In addition, the role-based access control for using the pod security policy is moved from cluster-scoped to namespace-scoped. This ensures that the triggers cannot use other pod security policies that are unrelated to a namespace.
  • Support for embedded trigger templates is now added. You can either use the name field to refer to an embedded template or embed the template inside the spec field.

2.1.3.2. Deprecated features

  • Pipeline templates that use PipelineResources CRDs are now deprecated and will be removed in a future release.
  • The template.name field is deprecated in favor of the template.ref field and will be removed in a future release.
  • The -c shorthand for the --check command has been removed. In addition, global tkn flags are added to the version command.

2.1.3.3. Known issues

  • CEL overlays add fields to a new top-level extensions function, instead of modifying the incoming event body. TriggerBinding resources can access values within this new extensions function using the $(extensions.<key>) syntax. Update your binding to use the $(extensions.<key>) syntax instead of the $(body.<overlay-key>) syntax.
  • The escaping parameters behavior by replacing " with \" is now removed. If you need to retain the old escaping parameters behavior add the tekton.dev/old-escape-quotes: true" annotation to your TriggerTemplate specification.
  • You can embed TriggerBinding resources by using the name and value fields inside a trigger or an event listener. However, you cannot specify both name and ref fields for a single binding. Use the ref field to refer to a TriggerBinding resource and the name field for embedded bindings.
  • An interceptor cannot attempt to reference a secret outside the namespace of an EventListener resource. You must include secrets in the namespace of the `EventListener`resource.
  • In Triggers 0.9.0 and later, if a body or header based TriggerBinding parameter is missing or malformed in an event payload, the default values are used instead of displaying an error.
  • Tasks and pipelines created with WhenExpression objects using Tekton Pipelines 0.16.x must be reapplied to fix their JSON annotations.
  • When a pipeline accepts an optional workspace and gives it to a task, the pipeline run stalls if the workspace is not provided.
  • To use the Buildah cluster task in a disconnected environment, ensure that the Dockerfile uses an internal image stream as the base image, and then use it in the same manner as any S2I cluster task.

2.1.3.4. Fixed issues

  • Extensions added by a CEL Interceptor are passed on to webhook interceptors by adding the Extensions field within the event body.
  • The activity timeout for log readers is now configurable using the LogOptions field. However, the default behavior of timeout in 10 seconds is retained.
  • The log command ignores the --follow flag when a task run or pipeline run is complete, and reads available logs instead of live logs.
  • References to the following Tekton resources: EventListener, TriggerBinding, ClusterTriggerBinding, Condition, and TriggerTemplate are now standardized and made consistent across all user-facing messages in tkn commands.
  • Previously, if you started a canceled task run or pipeline run with the --use-taskrun <canceled-task-run-name>, --use-pipelinerun <canceled-pipeline-run-name> or --last flags, the new run would be canceled. This bug is now fixed.
  • The tkn pr desc command is now enhanced to ensure that it does not fail in case of pipeline runs with conditions.
  • When you delete a task run using the tkn tr delete command with the --task option, and a cluster task exists with the same name, the task runs for the cluster task also get deleted. As a workaround, filter the task runs by using the TaskRefKind field.
  • The tkn triggertemplate describe command would display only part of the apiVersion value in the output. For example, only triggers.tekton.dev was displayed instead of triggers.tekton.dev/v1alpha1. This bug is now fixed.
  • The webhook, under certain conditions, would fail to acquire a lease and not function correctly. This bug is now fixed.
  • Pipelines with when expressions created in v0.16.3 can now be run in v0.17.1 and later. After an upgrade, you do not need to reapply pipeline definitions created in previous versions because both the uppercase and lowercase first letters for the annotations are now supported.
  • By default, the leader-election-ha field is now enabled for high availability. When the disable-ha controller flag is set to true, it disables high availability support.
  • Issues with duplicate cloud events are now fixed. Cloud events are now sent only when a condition changes the state, reason, or message.
  • When a service account name is missing from a PipelineRun or TaskRun spec, the controller uses the service account name from the config-defaults config map. If the service account name is also missing in the config-defaults config map, the controller now sets it to default in the spec.
  • Validation for compatibility with the affinity assistant is now supported when the same persistent volume claim is used for multiple workspaces, but with different subpaths.

2.1.4. Release notes for Red Hat OpenShift Pipelines Technology Preview 1.2

2.1.4.1. New features

Red Hat OpenShift Pipelines Technology Preview (TP) 1.2 is now available on OpenShift Container Platform 4.6. Red Hat OpenShift Pipelines TP 1.2 is updated to support:

  • Tekton Pipelines 0.16.3
  • Tekton tkn CLI 0.13.1
  • Tekton Triggers 0.8.1
  • cluster tasks based on Tekton Catalog 0.16
  • IBM Power Systems on OpenShift Container Platform 4.6
  • IBM Z and LinuxONE on OpenShift Container Platform 4.6

In addition to the fixes and stability improvements, the following sections highlight what is new in Red Hat OpenShift Pipelines 1.2.

2.1.4.1.1. Pipelines
  • This release of Red Hat OpenShift Pipelines adds support for a disconnected installation.

    Note

    Installations in restricted environments are currently not supported on IBM Power Systems, IBM Z, and LinuxONE.

  • You can now use the when field, instead of conditions resource, to run a task only when certain criteria are met. The key components of WhenExpression resources are Input, Operator, and Values. If all the when expressions evaluate to True, then the task is run. If any of the when expressions evaluate to False, the task is skipped.
  • Step statuses are now updated if a task run is canceled or times out.
  • Support for Git Large File Storage (LFS) is now available to build the base image used by git-init.
  • You can now use the taskSpec field to specify metadata, such as labels and annotations, when a task is embedded in a pipeline.
  • Cloud events are now supported by pipeline runs. Retries with backoff are now enabled for cloud events sent by the cloud event pipeline resource.
  • You can now set a default Workspace configuration for any workspace that a Task resource declares, but that a TaskRun resource does not explicitly provide.
  • Support is available for namespace variable interpolation for the PipelineRun namespace and TaskRun namespace.
  • Validation for TaskRun objects is now added to check that not more than one persistent volume claim workspace is used when a TaskRun resource is associated with an Affinity Assistant. If more than one persistent volume claim workspace is used, the task run fails with a TaskRunValidationFailed condition. Note that by default, the Affinity Assistant is disabled in Red Hat OpenShift Pipelines, so you will need to enable the assistant to use it.
2.1.4.1.2. Pipelines CLI
  • The tkn task describe, tkn taskrun describe, tkn clustertask describe, tkn pipeline describe, and tkn pipelinerun describe commands now:

    • Automatically select the Task, TaskRun, ClusterTask, Pipeline and PipelineRun resource, respectively, if only one of them is present.
    • Display the results of the Task, TaskRun, ClusterTask, Pipeline and PipelineRun resource in their outputs, respectively.
    • Display workspaces declared in the Task, TaskRun, ClusterTask, Pipeline and PipelineRun resource in their outputs, respectively.
  • You can now use the --prefix-name option with the tkn clustertask start command to specify a prefix for the name of a task run.
  • Interactive mode support has now been provided to the tkn clustertask start command.
  • You can now specify PodTemplate properties supported by pipelines using local or remote file definitions for TaskRun and PipelineRun objects.
  • You can now use the --use-params-defaults option with the tkn clustertask start command to use the default values set in the ClusterTask configuration and create the task run.
  • The --use-param-defaults flag for the tkn pipeline start command now prompts the interactive mode if the default values have not been specified for some of the parameters.
2.1.4.1.3. Triggers
  • The Common Expression Language (CEL) function named parseYAML has been added to parse a YAML string into a map of strings.
  • Error messages for parsing CEL expressions have been improved to make them more granular while evaluating expressions and when parsing the hook body for creating the evaluation environment.
  • Support is now available for marshaling boolean values and maps if they are used as the values of expressions in a CEL overlay mechanism.
  • The following fields have been added to the EventListener object:

    • The replicas field enables the event listener to run more than one pod by specifying the number of replicas in the YAML file.
    • The NodeSelector field enables the EventListener object to schedule the event listener pod to a specific node.
  • Webhook interceptors can now parse the EventListener-Request-URL header to extract parameters from the original request URL being handled by the event listener.
  • Annotations from the event listener can now be propagated to the deployment, services, and other pods. Note that custom annotations on services or deployment are overwritten, and hence, must be added to the event listener annotations so that they are propagated.
  • Proper validation for replicas in the EventListener specification is now available for cases when a user specifies the spec.replicas values as negative or zero.
  • You can now specify the TriggerCRD object inside the EventListener spec as a reference using the TriggerRef field to create the TriggerCRD object separately and then bind it inside the EventListener spec.
  • Validation and defaults for the TriggerCRD object are now available.

2.1.4.2. Deprecated features

  • $(params) parameters are now removed from the triggertemplate resource and replaced by $(tt.params) to avoid confusion between the resourcetemplate and triggertemplate resource parameters.
  • The ServiceAccount reference of the optional EventListenerTrigger-based authentication level has changed from an object reference to a ServiceAccountName string. This ensures that the ServiceAccount reference is in the same namespace as the EventListenerTrigger object.
  • The Conditions custom resource definition (CRD) is now deprecated; use the WhenExpressions CRD instead.
  • The PipelineRun.Spec.ServiceAccountNames object is being deprecated and replaced by the PipelineRun.Spec.TaskRunSpec[].ServiceAccountName object.

2.1.4.3. Known issues

  • This release of Red Hat OpenShift Pipelines adds support for a disconnected installation. However, some images used by the cluster tasks must be mirrored for them to work in disconnected clusters.
  • Pipelines in the openshift namespace are not deleted after you uninstall the Red Hat OpenShift Pipelines Operator. Use the oc delete pipelines -n openshift --all command to delete the pipelines.
  • Uninstalling the Red Hat OpenShift Pipelines Operator does not remove the event listeners.

    As a workaround, to remove the EventListener and Pod CRDs:

    1. Edit the EventListener object with the foregroundDeletion finalizers:

      $ oc patch el/<eventlistener_name> -p '{"metadata":{"finalizers":["foregroundDeletion"]}}' --type=merge

      For example:

      $ oc patch el/github-listener-interceptor -p '{"metadata":{"finalizers":["foregroundDeletion"]}}' --type=merge
    2. Delete the EventListener CRD:

      $ oc patch crd/eventlisteners.triggers.tekton.dev -p '{"metadata":{"finalizers":[]}}' --type=merge
  • When you run a multi-arch container image task without command specification on an IBM Power Systems (ppc64le) or IBM Z (s390x) cluster, the TaskRun resource fails with the following error:

    Error executing command: fork/exec /bin/bash: exec format error

    As a workaround, use an architecture specific container image or specify the sha256 digest to point to the correct architecture. To get the sha256 digest enter:

    $ skopeo inspect --raw <image_name>| jq '.manifests[] | select(.platform.architecture == "<architecture>") | .digest'

2.1.4.4. Fixed issues

  • A simple syntax validation to check the CEL filter, overlays in the Webhook validator, and the expressions in the interceptor has now been added.
  • Triggers no longer overwrite annotations set on the underlying deployment and service objects.
  • Previously, an event listener would stop accepting events. This fix adds an idle timeout of 120 seconds for the EventListener sink to resolve this issue.
  • Previously, canceling a pipeline run with a Failed(Canceled) state gave a success message. This has been fixed to display an error instead.
  • The tkn eventlistener list command now provides the status of the listed event listeners, thus enabling you to easily identify the available ones.
  • Consistent error messages are now displayed for the triggers list and triggers describe commands when triggers are not installed or when a resource cannot be found.
  • Previously, a large number of idle connections would build up during cloud event delivery. The DisableKeepAlives: true parameter was added to the cloudeventclient config to fix this issue. Thus, a new connection is set up for every cloud event.
  • Previously, the creds-init code would write empty files to the disk even if credentials of a given type were not provided. This fix modifies the creds-init code to write files for only those credentials that have actually been mounted from correctly annotated secrets.

2.1.5. Release notes for Red Hat OpenShift Pipelines Technology Preview 1.1

2.1.5.1. New features

Red Hat OpenShift Pipelines Technology Preview (TP) 1.1 is now available on OpenShift Container Platform 4.5. Red Hat OpenShift Pipelines TP 1.1 is updated to support:

  • Tekton Pipelines 0.14.3
  • Tekton tkn CLI 0.11.0
  • Tekton Triggers 0.6.1
  • cluster tasks based on Tekton Catalog 0.14

In addition to the fixes and stability improvements, the following sections highlight what is new in Red Hat OpenShift Pipelines 1.1.

2.1.5.1.1. Pipelines
  • Workspaces can now be used instead of pipeline resources. It is recommended that you use workspaces in OpenShift Pipelines, as pipeline resources are difficult to debug, limited in scope, and make tasks less reusable. For more details on workspaces, see the Understanding OpenShift Pipelines section.
  • Workspace support for volume claim templates has been added:

    • The volume claim template for a pipeline run and task run can now be added as a volume source for workspaces. The tekton-controller then creates a persistent volume claim (PVC) using the template that is seen as a PVC for all task runs in the pipeline. Thus you do not need to define the PVC configuration every time it binds a workspace that spans multiple tasks.
    • Support to find the name of the PVC when a volume claim template is used as a volume source is now available using variable substitution.
  • Support for improving audits:

    • The PipelineRun.Status field now contains the status of every task run in the pipeline and the pipeline specification used to instantiate a pipeline run to monitor the progress of the pipeline run.
    • Pipeline results have been added to the pipeline specification and PipelineRun status.
    • The TaskRun.Status field now contains the exact task specification used to instantiate the TaskRun resource.
  • Support to apply the default parameter to conditions.
  • A task run created by referencing a cluster task now adds the tekton.dev/clusterTask label instead of the tekton.dev/task label.
  • The kube config writer now adds the ClientKeyData and the ClientCertificateData configurations in the resource structure to enable replacement of the pipeline resource type cluster with the kubeconfig-creator task.
  • The names of the feature-flags and the config-defaults config maps are now customizable.
  • Support for the host network in the pod template used by the task run is now available.
  • An Affinity Assistant is now available to support node affinity in task runs that share workspace volume. By default, this is disabled on OpenShift Pipelines.
  • The pod template has been updated to specify imagePullSecrets to identify secrets that the container runtime should use to authorize container image pulls when starting a pod.
  • Support for emitting warning events from the task run controller if the controller fails to update the task run.
  • Standard or recommended k8s labels have been added to all resources to identify resources belonging to an application or component.
  • The Entrypoint process is now notified for signals and these signals are then propagated using a dedicated PID Group of the Entrypoint process.
  • The pod template can now be set on a task level at runtime using task run specs.
  • Support for emitting Kubernetes events:

    • The controller now emits events for additional task run lifecycle events - taskrun started and taskrun running.
    • The pipeline run controller now emits an event every time a pipeline starts.
  • In addition to the default Kubernetes events, support for cloud events for task runs is now available. The controller can be configured to send any task run events, such as create, started, and failed, as cloud events.
  • Support for using the $context.<task|taskRun|pipeline|pipelineRun>.name variable to reference the appropriate name when in pipeline runs and task runs.
  • Validation for pipeline run parameters is now available to ensure that all the parameters required by the pipeline are provided by the pipeline run. This also allows pipeline runs to provide extra parameters in addition to the required parameters.
  • You can now specify tasks within a pipeline that will always execute before the pipeline exits, either after finishing all tasks successfully or after a task in the pipeline failed, using the finally field in the pipeline YAML file.
  • The git-clone cluster task is now available.
2.1.5.1.2. Pipelines CLI
  • Support for embedded trigger binding is now available to the tkn evenlistener describe command.
  • Support to recommend subcommands and make suggestions if an incorrect subcommand is used.
  • The tkn task describe command now auto selects the task if only one task is present in the pipeline.
  • You can now start a task using default parameter values by specifying the --use-param-defaults flag in the tkn task start command.
  • You can now specify a volume claim template for pipeline runs or task runs using the --workspace option with the tkn pipeline start or tkn task start commands.
  • The tkn pipelinerun logs command now displays logs for the final tasks listed in the finally section.
  • Interactive mode support has now been provided to the tkn task start command and the describe subcommand for the following tkn resources: pipeline, pipelinerun, task, taskrun, clustertask, and pipelineresource.
  • The tkn version command now displays the version of the triggers installed in the cluster.
  • The tkn pipeline describe command now displays parameter values and timeouts specified for tasks used in the pipeline.
  • Support added for the --last option for the tkn pipelinerun describe and the tkn taskrun describe commands to describe the most recent pipeline run or task run, respectively.
  • The tkn pipeline describe command now displays the conditions applicable to the tasks in the pipeline.
  • You can now use the --no-headers and --all-namespaces flags with the tkn resource list command.
2.1.5.1.3. Triggers
  • The following Common Expression Language (CEL) functions are now available:

    • parseURL to parse and extract portions of a URL
    • parseJSON to parse JSON value types embedded in a string in the payload field of the deployment webhook
  • A new interceptor for webhooks from Bitbucket has been added.
  • Event listeners now display the Address URL and the Available status as additional fields when listed with the kubectl get command.
  • trigger template params now use the $(tt.params.<paramName>) syntax instead of $(params.<paramName>) to reduce the confusion between trigger template and resource templates params.
  • You can now add tolerations in the EventListener CRD to ensure that event listeners are deployed with the same configuration even if all nodes are tainted due to security or management issues.
  • You can now add a Readiness Probe for event listener Deployment at URL/live.
  • Support for embedding TriggerBinding specifications in event listener triggers is now added.
  • Trigger resources are now annotated with the recommended app.kubernetes.io labels.

2.1.5.2. Deprecated features

The following items are deprecated in this release:

  • The --namespace or -n flags for all cluster-wide commands, including the clustertask and clustertriggerbinding commands, are deprecated. It will be removed in a future release.
  • The name field in triggers.bindings within an event listener has been deprecated in favor of the ref field and will be removed in a future release.
  • Variable interpolation in trigger templates using $(params) has been deprecated in favor of using $(tt.params) to reduce confusion with the pipeline variable interpolation syntax. The $(params.<paramName>) syntax will be removed in a future release.
  • The tekton.dev/task label is deprecated on cluster tasks.
  • The TaskRun.Status.ResourceResults.ResourceRef field is deprecated and will be removed.
  • The tkn pipeline create, tkn task create, and tkn resource create -f subcommands have been removed.
  • Namespace validation has been removed from tkn commands.
  • The default timeout of 1h and the -t flag for the tkn ct start command have been removed.
  • The s2i cluster task has been deprecated.

2.1.5.3. Known issues

  • Conditions do not support workspaces.
  • The --workspace option and the interactive mode is not supported for the tkn clustertask start command.
  • Support of backward compatibility for $(params.<paramName>) syntax forces you to use trigger templates with pipeline specific params as the trigger s webhook is unable to differentiate trigger params from pipelines params.
  • Pipeline metrics report incorrect values when you run a promQL query for tekton_taskrun_count and tekton_taskrun_duration_seconds_count.
  • pipeline runs and task runs continue to be in the Running and Running(Pending) states respectively even when a non existing PVC name is given to a workspace.

2.1.5.4. Fixed issues

  • Previously, the tkn task delete <name> --trs command would delete both the task and cluster task if the name of the task and cluster task were the same. With this fix, the command deletes only the task runs that are created by the task <name>.
  • Previously the tkn pr delete -p <name> --keep 2 command would disregard the -p flag when used with the --keep flag and would delete all the pipeline runs except the latest two. With this fix, the command deletes only the pipeline runs that are created by the pipeline <name>, except for the latest two.
  • The tkn triggertemplate describe output now displays resource templates in a table format instead of YAML format.
  • Previously the buildah cluster task failed when a new user was added to a container. With this fix, the issue has been resolved.

2.1.6. Release notes for Red Hat OpenShift Pipelines Technology Preview 1.0

2.1.6.1. New features

Red Hat OpenShift Pipelines Technology Preview (TP) 1.0 is now available on OpenShift Container Platform 4.4. Red Hat OpenShift Pipelines TP 1.0 is updated to support:

  • Tekton Pipelines 0.11.3
  • Tekton tkn CLI 0.9.0
  • Tekton Triggers 0.4.0
  • cluster tasks based on Tekton Catalog 0.11

In addition to the fixes and stability improvements, the following sections highlight what is new in Red Hat OpenShift Pipelines 1.0.

2.1.6.1.1. Pipelines
  • Support for v1beta1 API Version.
  • Support for an improved limit range. Previously, limit range was specified exclusively for the task run and the pipeline run. Now there is no need to explicitly specify the limit range. The minimum limit range across the namespace is used.
  • Support for sharing data between tasks using task results and task params.
  • Pipelines can now be configured to not overwrite the HOME environment variable and the working directory of steps.
  • Similar to task steps, sidecars now support script mode.
  • You can now specify a different scheduler name in task run podTemplate resource.
  • Support for variable substitution using Star Array Notation.
  • Tekton controller can now be configured to monitor an individual namespace.
  • A new description field is now added to the specification of pipelines, tasks, cluster tasks, resources, and conditions.
  • Addition of proxy parameters to Git pipeline resources.
2.1.6.1.2. Pipelines CLI
  • The describe subcommand is now added for the following tkn resources: EventListener, Condition, TriggerTemplate, ClusterTask, and TriggerSBinding.
  • Support added for v1beta1 to the following resources along with backward compatibility for v1alpha1: ClusterTask, Task, Pipeline, PipelineRun, and TaskRun.
  • The following commands can now list output from all namespaces using the --all-namespaces flag option: tkn task list, tkn pipeline list, tkn taskrun list, tkn pipelinerun list

    The output of these commands is also enhanced to display information without headers using the --no-headers flag option.

  • You can now start a pipeline using default parameter values by specifying --use-param-defaults flag in the tkn pipelines start command.
  • Support for workspace is now added to tkn pipeline start and tkn task start commands.
  • A new clustertriggerbinding command is now added with the following subcommands: describe, delete, and list.
  • You can now directly start a pipeline run using a local or remote yaml file.
  • The describe subcommand now displays an enhanced and detailed output. With the addition of new fields, such as description, timeout, param description, and sidecar status, the command output now provides more detailed information about a specific tkn resource.
  • The tkn task log command now displays logs directly if only one task is present in the namespace.
2.1.6.1.3. Triggers
  • Triggers can now create both v1alpha1 and v1beta1 pipeline resources.
  • Support for new Common Expression Language (CEL) interceptor function - compareSecret. This function securely compares strings to secrets in CEL expressions.
  • Support for authentication and authorization at the event listener trigger level.

2.1.6.2. Deprecated features

The following items are deprecated in this release:

  • The environment variable $HOME, and variable workingDir in the Steps specification are deprecated and might be changed in a future release. Currently in a Step container, the HOME and workingDir variables are overwritten to /tekton/home and /workspace variables, respectively.

    In a later release, these two fields will not be modified, and will be set to values defined in the container image and the Task YAML. For this release, use the disable-home-env-overwrite and disable-working-directory-overwrite flags to disable overwriting of the HOME and workingDir variables.

  • The following commands are deprecated and might be removed in the future release: tkn pipeline create, tkn task create.
  • The -f flag with the tkn resource create command is now deprecated. It might be removed in the future release.
  • The -t flag and the --timeout flag (with seconds format) for the tkn clustertask create command are now deprecated. Only duration timeout format is now supported, for example 1h30s. These deprecated flags might be removed in the future release.

2.1.6.3. Known issues

  • If you are upgrading from an older version of Red Hat OpenShift Pipelines, you must delete your existing deployments before upgrading to Red Hat OpenShift Pipelines version 1.0. To delete an existing deployment, you must first delete Custom Resources and then uninstall the Red Hat OpenShift Pipelines Operator. For more details, see the uninstalling Red Hat OpenShift Pipelines section.
  • Submitting the same v1alpha1 tasks more than once results in an error. Use the oc replace command instead of oc apply when re-submitting a v1alpha1 task.
  • The buildah cluster task does not work when a new user is added to a container.

    When the Operator is installed, the --storage-driver flag for the buildah cluster task is not specified, therefore the flag is set to its default value. In some cases, this causes the storage driver to be set incorrectly. When a new user is added, the incorrect storage-driver results in the failure of the buildah cluster task with the following error:

    useradd: /etc/passwd.8: lock file already used
    useradd: cannot lock /etc/passwd; try again later.

    As a workaround, manually set the --storage-driver flag value to overlay in the buildah-task.yaml file:

    1. Login to your cluster as a cluster-admin:

      $ oc login -u <login> -p <password> https://openshift.example.com:6443
    2. Use the oc edit command to edit buildah cluster task:

      $ oc edit clustertask buildah

      The current version of the buildah clustertask YAML file opens in the editor set by your EDITOR environment variable.

    3. Under the Steps field, locate the following command field:

       command: ['buildah', 'bud', '--format=$(params.FORMAT)', '--tls-verify=$(params.TLSVERIFY)', '--layers', '-f', '$(params.DOCKERFILE)', '-t', '$(resources.outputs.image.url)', '$(params.CONTEXT)']
    4. Replace the command field with the following:

       command: ['buildah', '--storage-driver=overlay', 'bud', '--format=$(params.FORMAT)', '--tls-verify=$(params.TLSVERIFY)', '--no-cache', '-f', '$(params.DOCKERFILE)', '-t', '$(params.IMAGE)', '$(params.CONTEXT)']
    5. Save the file and exit.

    Alternatively, you can also modify the buildah cluster task YAML file directly on the web console by navigating to PipelinesCluster Tasksbuildah. Select Edit Cluster Task from the Actions menu and replace the command field as shown in the previous procedure.

2.1.6.4. Fixed issues

  • Previously, the DeploymentConfig task triggered a new deployment build even when an image build was already in progress. This caused the deployment of the pipeline to fail. With this fix, the deploy task command is now replaced with the oc rollout status command which waits for the in-progress deployment to finish.
  • Support for APP_NAME parameter is now added in pipeline templates.
  • Previously, the pipeline template for Java S2I failed to look up the image in the registry. With this fix, the image is looked up using the existing image pipeline resources instead of the user provided IMAGE_NAME parameter.
  • All the OpenShift Pipelines images are now based on the Red Hat Universal Base Images (UBI).
  • Previously, when the pipeline was installed in a namespace other than tekton-pipelines, the tkn version command displayed the pipeline version as unknown. With this fix, the tkn version command now displays the correct pipeline version in any namespace.
  • The -c flag is no longer supported for the tkn version command.
  • Non-admin users can now list the cluster trigger bindings.
  • The event listener CompareSecret function is now fixed for the CEL Interceptor.
  • The list, describe, and start subcommands for tasks and cluster tasks now correctly display the output in case a task and cluster task have the same name.
  • Previously, the OpenShift Pipelines Operator modified the privileged security context constraints (SCCs), which caused an error during cluster upgrade. This error is now fixed.
  • In the tekton-pipelines namespace, the timeouts of all task runs and pipeline runs are now set to the value of default-timeout-minutes field using the config map.
  • Previously, the pipelines section in the web console was not displayed for non-admin users. This issue is now resolved.

2.2. Understanding OpenShift Pipelines

Red Hat OpenShift Pipelines is a cloud-native, continuous integration and continuous delivery (CI/CD) solution based on Kubernetes resources. It uses Tekton building blocks to automate deployments across multiple platforms by abstracting away the underlying implementation details. Tekton introduces a number of standard custom resource definitions (CRDs) for defining CI/CD pipelines that are portable across Kubernetes distributions.

2.2.1. Key features

  • Red Hat OpenShift Pipelines is a serverless CI/CD system that runs pipelines with all the required dependencies in isolated containers.
  • Red Hat OpenShift Pipelines are designed for decentralized teams that work on microservice-based architecture.
  • Red Hat OpenShift Pipelines use standard CI/CD pipeline definitions that are easy to extend and integrate with the existing Kubernetes tools, enabling you to scale on-demand.
  • You can use Red Hat OpenShift Pipelines to build images with Kubernetes tools such as Source-to-Image (S2I), Buildah, Buildpacks, and Kaniko that are portable across any Kubernetes platform.
  • You can use the OpenShift Container Platform Developer console to create Tekton resources, view logs of pipeline runs, and manage pipelines in your OpenShift Container Platform namespaces.

2.2.2. OpenShift Pipeline Concepts

This guide provides a detailed view of the various pipeline concepts.

2.2.2.1. Tasks

Tasks are the building blocks of a Pipeline and consists of sequentially executed steps. It is essentially a function of inputs and outputs. A Task can run individually or as a part of the pipeline. Tasks are reusable and can be used in multiple Pipelines.

Steps are a series of commands that are sequentially executed by the Task and achieve a specific goal, such as building an image. Every Task runs as a pod, and each Step runs as a container within that pod. Because Steps run within the same pod, they can access the same volumes for caching files, config maps, and secrets.

The following example shows the apply-manifests Task.

apiVersion: tekton.dev/v1beta1 1
kind: Task 2
metadata:
  name: apply-manifests 3
spec: 4
  workspaces:
  - name: source
  params:
    - name: manifest_dir
      description: The directory in source that contains yaml manifests
      type: string
      default: "k8s"
  steps:
    - name: apply
      image: image-registry.openshift-image-registry.svc:5000/openshift/cli:latest
      workingDir: /workspace/source
      command: ["/bin/bash", "-c"]
      args:
        - |-
          echo Applying manifests in $(params.manifest_dir) directory
          oc apply -f $(params.manifest_dir)
          echo -----------------------------------
1
The Task API version, v1beta1.
2
The type of Kubernetes object, Task.
3
The unique name of this Task.
4
The list of parameters and Steps in the Task and the workspace used by the Task.

This Task starts the pod and runs a container inside that pod using the specified image to run the specified commands.

2.2.2.2. TaskRun

A TaskRun instantiates a Task for execution with specific inputs, outputs, and execution parameters on a cluster. It can be invoked on its own or as part of a PipelineRun for each Task in a pipeline.

A Task consists of one or more Steps that execute container images, and each container image performs a specific piece of build work. A TaskRun executes the Steps in a Task in the specified order, until all Steps execute successfully or a failure occurs. A TaskRun is automatically created by a PipelineRun for each Task in a Pipeline.

The following example shows a TaskRun that runs the apply-manifests Task with the relevant input parameters:

apiVersion: tekton.dev/v1beta1 1
kind: TaskRun 2
metadata:
  name: apply-manifests-taskrun 3
spec: 4
  serviceAccountName: pipeline
  taskRef: 5
    kind: Task
    name: apply-manifests
  workspaces: 6
  - name: source
    persistentVolumeClaim:
      claimName: source-pvc
1
TaskRun API version v1beta1.
2
Specifies the type of Kubernetes object. In this example, TaskRun.
3
Unique name to identify this TaskRun.
4
Definition of the TaskRun. For this TaskRun, the Task and the required workspace are specified.
5
Name of the Task reference used for this TaskRun. This TaskRun executes the apply-manifests Task.
6
Workspace used by the TaskRun.

2.2.2.3. Pipelines

A Pipeline is a collection of Task resources arranged in a specific order of execution. They are executed to construct complex workflows that automate the build, deployment and delivery of applications. You can define a CI/CD workflow for your application using pipelines containing one or more tasks.

A Pipeline resource definition consists of a number of fields or attributes, which together enable the pipeline to accomplish a specific goal. Each Pipeline resource definition must contain at least one Task resource, which ingests specific inputs and produces specific outputs. The pipeline definition can also optionally include Conditions, Workspaces, Parameters, or Resources depending on the application requirements.

The following example shows the build-and-deploy pipeline, which builds an application image from a Git repository using the buildah ClusterTask resource:

apiVersion: tekton.dev/v1beta1 1
kind: Pipeline 2
metadata:
  name: build-and-deploy 3
spec: 4
  workspaces: 5
  - name: shared-workspace
  params: 6
  - name: deployment-name
    type: string
    description: name of the deployment to be patched
  - name: git-url
    type: string
    description: url of the git repo for the code of deployment
  - name: git-revision
    type: string
    description: revision to be used from repo of the code for deployment
    default: "pipelines-1.4"
  - name: IMAGE
    type: string
    description: image to be built from the code
  tasks: 7
  - name: fetch-repository
    taskRef:
      name: git-clone
      kind: ClusterTask
    workspaces:
    - name: output
      workspace: shared-workspace
    params:
    - name: url
      value: $(params.git-url)
    - name: subdirectory
      value: ""
    - name: deleteExisting
      value: "true"
    - name: revision
      value: $(params.git-revision)
  - name: build-image 8
    taskRef:
      name: buildah
      kind: ClusterTask
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter:
    - fetch-repository
  - name: apply-manifests 9
    taskRef:
      name: apply-manifests
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter: 10
    - build-image
  - name: update-deployment
    taskRef:
      name: update-deployment
    workspaces:
    - name: source
      workspace: shared-workspace
    params:
    - name: deployment
      value: $(params.deployment-name)
    - name: IMAGE
      value: $(params.IMAGE)
    runAfter:
    - apply-manifests
1
Pipeline API version v1beta1.
2
Specifies the type of Kubernetes object. In this example, Pipeline.
3
Unique name of this Pipeline.
4
Specifies the definition and structure of the Pipeline.
5
Workspaces used across all the Tasks in the Pipeline.
6
Parameters used across all the Tasks in the Pipeline.
7
Specifies the list of Tasks used in the Pipeline.
8
Task build-image, which uses the buildah ClusterTask to build application images from a given Git repository.
9
Task apply-manifests, which uses a user-defined Task with the same name.
10
Specifies the sequence in which Tasks are run in a Pipeline. In this example, the apply-manifests Task is run only after the build-image Task is completed.

2.2.2.4. PipelineRun

A PipelineRun is the running instance of a Pipeline. It instantiates a Pipeline for execution with specific inputs, outputs, and execution parameters on a cluster. A corresponding TaskRun is created for each Task automatically in the PipelineRun.

All the Tasks in the Pipeline are executed in the defined sequence until all Tasks are successful or a Task fails. The status field tracks and stores the progress of each TaskRun in the PipelineRun for monitoring and auditing purpose.

The following example shows a PipelineRun to run the build-and-deploy Pipeline with relevant resources and parameters:

apiVersion: tekton.dev/v1beta1 1
kind: PipelineRun 2
metadata:
  name: build-deploy-api-pipelinerun 3
spec:
  pipelineRef:
    name: build-and-deploy 4
  params: 5
  - name: deployment-name
    value: vote-api
  - name: git-url
    value: https://github.com/openshift-pipelines/vote-api.git
  - name: IMAGE
    value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api
  workspaces: 6
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi
1
PipelineRun API version v1beta1.
2
Specifies the type of Kubernetes object. In this example, PipelineRun.
3
Unique name to identify this PipelineRun.
4
Name of the Pipeline to be run. In this example, build-and-deploy.
5
Specifies the list of parameters required to run the Pipeline.
6
Workspace used by the PipelineRun.

2.2.2.5. Workspaces

Note

It is recommended that you use Workspaces instead of PipelineResources in OpenShift Pipelines, as PipelineResources are difficult to debug, limited in scope, and make Tasks less reusable.

Workspaces declare shared storage volumes that a Task in a Pipeline needs at runtime to receive input or provide output. Instead of specifying the actual location of the volumes, Workspaces enable you to declare the filesystem or parts of the filesystem that would be required at runtime. A Task or Pipeline declares the Workspace and you must provide the specific location details of the volume. It is then mounted into that Workspace in a TaskRun or a PipelineRun. This separation of volume declaration from runtime storage volumes makes the Tasks reusable, flexible, and independent of the user environment.

With Workspaces, you can:

  • Store Task inputs and outputs
  • Share data among Tasks
  • Use it as a mount point for credentials held in Secrets
  • Use it as a mount point for configurations held in ConfigMaps
  • Use it as a mount point for common tools shared by an organization
  • Create a cache of build artifacts that speed up jobs

You can specify Workspaces in the TaskRun or PipelineRun using:

  • A read-only ConfigMaps or Secret
  • An existing PersistentVolumeClaim shared with other Tasks
  • A PersistentVolumeClaim from a provided VolumeClaimTemplate
  • An emptyDir that is discarded when the TaskRun completes

The following example shows a code snippet of the build-and-deploy Pipeline, which declares a shared-workspace Workspace for the build-image and apply-manifests Tasks as defined in the Pipeline.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: build-and-deploy
spec:
  workspaces: 1
  - name: shared-workspace
  params:
...
  tasks: 2
  - name: build-image
    taskRef:
      name: buildah
      kind: ClusterTask
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    workspaces: 3
    - name: source 4
      workspace: shared-workspace 5
    runAfter:
    - fetch-repository
  - name: apply-manifests
    taskRef:
      name: apply-manifests
    workspaces: 6
    - name: source
      workspace: shared-workspace
    runAfter:
      - build-image
...
1
List of Workspaces shared between the Tasks defined in the Pipeline. A Pipeline can define as many Workspaces as required. In this example, only one Workspace named shared-workspace is declared.
2
Definition of Tasks used in the Pipeline. This snippet defines two Tasks, build-image and apply-manifests, which share a common Workspace.
3
List of Workspaces used in the build-image Task. A Task definition can include as many Workspaces as it requires. However, it is recommended that a Task uses at most one writable Workspace.
4
Name that uniquely identifies the Workspace used in the Task. This Task uses one Workspace named source.
5
Name of the Pipeline Workspace used by the Task. Note that the Workspace source in turn uses the Pipeline Workspace named shared-workspace.
6
List of Workspaces used in the apply-manifests Task. Note that this Task shares the source Workspace with the build-image Task.

Workspaces help tasks share data, and allow you to specify one or more volumes that each task in the pipeline requires during execution. You can create a persistent volume claim or provide a volume claim template that creates a persistent volume claim for you.

The following code snippet of the build-deploy-api-pipelinerun PipelineRun uses a volume claim template to create a persistent volume claim for defining the storage volume for the shared-workspace Workspace used in the build-and-deploy Pipeline.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: build-deploy-api-pipelinerun
spec:
  pipelineRef:
    name: build-and-deploy
  params:
...

  workspaces: 1
  - name: shared-workspace 2
    volumeClaimTemplate: 3
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi
1
Specifies the list of Pipeline Workspaces for which volume binding will be provided in the PipelineRun.
2
The name of the Workspace in the Pipeline for which the volume is being provided.
3
Specifies a volume claim template that creates a persistent volume claim to define the storage volume for the workspace.

2.2.2.6. Triggers

Use Triggers in conjunction with pipelines to create a full-fledged CI/CD system where Kubernetes resources define the entire CI/CD execution. Triggers capture the external events, such as a Git pull request, and process them to extract key pieces of information. Mapping this event data to a set of predefined parameters triggers a series of tasks that can then create and deploy Kubernetes resources and instantiate the pipeline.

For example, you define a CI/CD workflow using Red Hat OpenShift Pipelines for your application. The pipeline must start for any new changes to take effect in the application repository. Triggers automate this process by capturing and processing any change event and by triggering a pipeline run that deploys the new image with the latest changes.

Triggers consist of the following main resources that work together to form a reusable, decoupled, and self-sustaining CI/CD system:

  • The TriggerBinding resource validates events, extracts the fields from an event payload, and stores them as parameters.

    The following example shows a code snippet of the TriggerBinding resource, which extracts the Git repository information from the received event payload:

    apiVersion: triggers.tekton.dev/v1alpha1 1
    kind: TriggerBinding 2
    metadata:
      name: vote-app 3
    spec:
      params: 4
      - name: git-repo-url
        value: $(body.repository.url)
      - name: git-repo-name
        value: $(body.repository.name)
      - name: git-revision
        value: $(body.head_commit.id)
    1
    The API version of the TriggerBinding resource. In this example, v1alpha1.
    2
    Specifies the type of Kubernetes object. In this example, TriggerBinding.
    3
    Unique name to identify the TriggerBinding resource.
    4
    List of parameters which will be extracted from the received event payload and passed to the TriggerTemplate resource. In this example, the Git repository URL, name, and revision are extracted from the body of the event payload.
  • The TriggerTemplate resource acts as a standard for the way resources must be created. It specifies the way parameterized data from the TriggerBinding resource should be used. A trigger template receives input from the trigger binding, and then performs a series of actions that results in creation of new pipeline resources, and initiation of a new pipeline run.

    The following example shows a code snippet of a TriggerTemplate resource, which creates a pipeline run using the Git repository information received from the TriggerBinding resource you just created:

    apiVersion: triggers.tekton.dev/v1alpha1 1
    kind: TriggerTemplate 2
    metadata:
      name: vote-app 3
    spec:
      params: 4
      - name: git-repo-url
        description: The git repository url
      - name: git-revision
        description: The git revision
        default: pipelines-1.4
      - name: git-repo-name
        description: The name of the deployment to be created / patched
    
      resourcetemplates: 5
      - apiVersion: tekton.dev/v1beta1
        kind: PipelineRun
        metadata:
          name: build-deploy-$(tt.params.git-repo-name)-$(uid)
        spec:
          serviceAccountName: pipeline
          pipelineRef:
            name: build-and-deploy
          params:
          - name: deployment-name
            value: $(tt.params.git-repo-name)
          - name: git-url
            value: $(tt.params.git-repo-url)
          - name: git-revision
            value: $(tt.params.git-revision)
          - name: IMAGE
            value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/$(tt.params.git-repo-name)
          workspaces:
          - name: shared-workspace
            volumeClaimTemplate:
             spec:
              accessModes:
               - ReadWriteOnce
              resources:
                requests:
                  storage: 500Mi
    1
    The API version of the TriggerTemplate resource. In this example, v1alpha1.
    2
    Specifies the type of Kubernetes object. In this example, TriggerTemplate.
    3
    Unique name to identify the TriggerTemplate resource.
    4
    Parameters supplied by the TriggerBinding or EventListerner resources.
    5
    List of templates that specify the way resources must be created using the parameters received through the TriggerBinding or EventListener resources.
  • The Trigger resource connects the TriggerBinding and TriggerTemplate resources, and this Trigger resource is referenced in the EventListener specification.

    The following example shows a code snippet of a Trigger resource, named vote-trigger that connects the TriggerBinding and TriggerTemplate resources.

    apiVersion: triggers.tekton.dev/v1alpha1 1
    kind: Trigger 2
    metadata:
      name: vote-trigger 3
    spec:
      serviceAccountName: pipeline 4
      bindings:
        - ref: vote-app 5
      template: 6
         ref: vote-app
    1
    The API version of the Trigger resource. In this example, v1alpha1.
    2
    Specifies the type of Kubernetes object. In this example, Trigger.
    3
    Unique name to identify the Trigger resource.
    4
    Service account name to be used.
    5
    Name of the TriggerBinding resource to be connected to the TriggerTemplate resource.
    6
    Name of the TriggerTemplate resource to be connected to the TriggerBinding resource.
  • The EventListener resource provides an endpoint, or an event sink, that listens for incoming HTTP-based events with a JSON payload. It extracts event parameters from each TriggerBinding resource, and then processes this data to create Kubernetes resources as specified by the corresponding TriggerTemplate resource. The EventListener resource also performs lightweight event processing or basic filtering on the payload using event interceptors, which identify the type of payload and optionally modify it. Currently, pipeline triggers support four types of interceptors: Webhook Interceptors, GitHub Interceptors, GitLab Interceptors, and Common Expression Language (CEL) Interceptors.

    The following example shows an EventListener resource, which references the Trigger resource named vote-trigger.

    apiVersion: triggers.tekton.dev/v1alpha1 1
    kind: EventListener 2
    metadata:
      name: vote-app 3
    spec:
      serviceAccountName: pipeline 4
      triggers:
        - triggerRef: vote-trigger 5
    1
    The API version of the EventListener resource. In this example, v1alpha1.
    2
    Specifies the type of Kubernetes object. In this example, EventListener.
    3
    Unique name to identify the EventListener resource.
    4
    Service account name to be used.
    5
    Name of the Trigger resource referenced by the EventListener resource.

Triggers in Red Hat OpenShift Pipelines support both HTTP (insecure) and HTTPS (secure HTTP) connections to the Eventlistener resource. With the secure HTTPS connection, you get end-to-end secure connection within and outside the cluster. After you create a namespace, you can enable this secure HTTPS connection for the Eventlistener resource by adding the operator.tekton.dev/enable-annotation=enabled label to the namespace, and then creating a Trigger resource and a secured route using re-encrypt TLS termination.

2.2.3. Additional resources

2.3. Installing OpenShift Pipelines

This guide walks cluster administrators through the process of installing the Red Hat OpenShift Pipelines Operator to an OpenShift Container Platform cluster.

Prerequisites

  • You have access to an OpenShift Container Platform cluster using an account with cluster-admin permissions.
  • You have installed oc CLI.
  • You have installed OpenShift Pipelines (tkn) CLI on your local system.

2.3.1. Installing the Red Hat OpenShift Pipelines Operator in web console

You can install Red Hat OpenShift Pipelines using the Operator listed in the OpenShift Container Platform OperatorHub. When you install the Red Hat OpenShift Pipelines Operator, the custom resources (CRs) required for the pipelines configuration are automatically installed along with the Operator.

The default Operator custom resource definition (CRD) config.operator.tekton.dev is now replaced by tektonconfigs.operator.tekton.dev. In addition, the Operator provides the following additional CRDs to individually manage OpenShift Pipelines components: tektonpipelines.operator.tekton.dev, tektontriggers.operator.tekton.dev and tektonaddons.operator.tekton.dev.

If you have OpenShift Pipelines already installed on your cluster, the existing installation is seamlessly upgraded. The Operator will replace the instance of config.operator.tekton.dev on your cluster with an instance of tektonconfigs.operator.tekton.dev and additional objects of the other CRDs as necessary.

Warning

If you manually changed your existing installation, such as, changing the target namespace in the config.operator.tekton.dev CRD instance by making changes to the resource name - cluster field, then the upgrade path is not smooth. In such cases, the recommended workflow is to uninstall your installation and reinstall the Red Hat OpenShift Pipelines Operator.

The Red Hat OpenShift Pipelines Operator now provides the option to choose the components that you want to install by specifying profiles as part of the TektonConfig CR. The TektonConfig CR is automatically installed when the Operator is installed. The supported profiles are:

  • Basic: This installs only Tekton pipelines.
  • Default: This installs Tekton pipeline and Tekton triggers.
  • All: This is the default profile used when the TektonConfig CR is installed. This profile installs all of the Tekton components: Tekton Pipelines, Tekton Triggers, Tekton Addons(which include ClusterTasks, ClusterTriggerBindings, ConsoleCLIDownload, ConsoleQuickStart and ConsoleYAMLSample resources).

Procedure

  1. In the Administrator perspective of the web console, navigate to OperatorsOperatorHub.
  2. Use the Filter by keyword box to search for Red Hat OpenShift Pipelines Operator in the catalog. Click the Red Hat OpenShift Pipelines Operator tile.
  3. Read the brief description about the Operator on the Red Hat OpenShift Pipelines Operator page. Click Install.
  4. On the Install Operator page:

    1. Select All namespaces on the cluster (default) for the Installation Mode. This mode installs the Operator in the default openshift-operators namespace, which enables the Operator to watch and be made available to all namespaces in the cluster.
    2. Select Automatic for the Approval Strategy. This ensures that the future upgrades to the Operator are handled automatically by the Operator Lifecycle Manager (OLM). If you select the Manual approval strategy, OLM creates an update request. As a cluster administrator, you must then manually approve the OLM update request to update the Operator to the new version.
    3. Select an Update Channel.

      • The Stable channel enables installation of the latest stable and supported release of the Red Hat OpenShift Pipelines Operator.
      • The preview channel enables installation of the latest preview version of the Red Hat OpenShift Pipelines Operator, which may contain features that are not yet available from the Stable channel and is not supported.
  5. Click Install. You will see the Operator listed on the Installed Operators page.

    Note

    The Operator is installed automatically into the openshift-operators namespace.

  6. Verify that the Status is set to Succeeded Up to date to confirm successful installation of Red Hat OpenShift Pipelines Operator.

2.3.2. Installing the OpenShift Pipelines Operator using the CLI

You can install Red Hat OpenShift Pipelines Operator from the OperatorHub using the CLI.

Procedure

  1. Create a Subscription object YAML file to subscribe a namespace to the Red Hat OpenShift Pipelines Operator, for example, sub.yaml:

    Example Subscription

    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: openshift-pipelines-operator
      namespace: openshift-operators
    spec:
      channel:  <channel name> 1
      name: openshift-pipelines-operator-rh 2
      source: redhat-operators 3
      sourceNamespace: openshift-marketplace 4

    1
    Specify the channel name from where you want to subscribe the Operator
    2
    Name of the Operator to subscribe to.
    3
    Name of the CatalogSource that provides the Operator.
    4
    Namespace of the CatalogSource. Use openshift-marketplace for the default OperatorHub CatalogSources.
  2. Create the Subscription object:

    $ oc apply -f sub.yaml

    The Red Hat OpenShift Pipelines Operator is now installed in the default target namespace openshift-operators.

2.3.3. Red Hat OpenShift Pipelines Operator in a restricted environment

The Red Hat OpenShift Pipelines Operator enables support for installation of pipelines in a restricted network environment.

The Operator installs a proxy webhook that sets the proxy environment variables in the containers of the pod created by tekton-controllers based on the cluster proxy object. It also sets the proxy environment variables in the TektonPipelines, TektonTriggers, Controllers, Webhooks, and Operator Proxy Webhook resources.

By default, the proxy webhook is disabled for the openshift-pipelines namespace. To disable it for any other namespace, you can add the operator.tekton.dev/disable-proxy: true label to the namespace object.

2.3.4. Additional Resources

2.4. Uninstalling OpenShift Pipelines

Uninstalling the Red Hat OpenShift Pipelines Operator is a two-step process:

  1. Delete the Custom Resources (CRs) that were added by default when you installed the Red Hat OpenShift Pipelines Operator.
  2. Uninstall the Red Hat OpenShift Pipelines Operator.

Uninstalling only the Operator will not remove the Red Hat OpenShift Pipelines components created by default when the Operator is installed.

2.4.1. Deleting the Red Hat OpenShift Pipelines components and Custom Resources

Delete the Custom Resources (CRs) created by default during installation of the Red Hat OpenShift Pipelines Operator.

Procedure

  1. In the Administrator perspective of the web console, navigate to AdministrationCustom Resource Definition.
  2. Type config.operator.tekton.dev in the Filter by name box to search for the Red Hat OpenShift Pipelines Operator CRs.
  3. Click CRD Config to see the Custom Resource Definition Details page.
  4. Click the Actions drop-down menu and select Delete Custom Resource Definition.

    Note

    Deleting the CRs will delete the Red Hat OpenShift Pipelines components, and all the Tasks and Pipelines on the cluster will be lost.

  5. Click Delete to confirm the deletion of the CRs.

2.4.2. Uninstalling the Red Hat OpenShift Pipelines Operator

Procedure

  1. From the OperatorsOperatorHub page, use the Filter by keyword box to search for Red Hat OpenShift Pipelines Operator.
  2. Click the OpenShift Pipelines Operator tile. The Operator tile indicates it is installed.
  3. In the OpenShift Pipelines Operator descriptor page, click Uninstall.

Additional Resources

2.5. Creating CI/CD solutions for applications using OpenShift Pipelines

With Red Hat OpenShift Pipelines, you can create a customized CI/CD solution to build, test, and deploy your application.

To create a full-fledged, self-serving CI/CD pipeline for an application, perform the following tasks:

  • Create custom tasks, or install existing reusable tasks.
  • Create and define the delivery pipeline for your application.
  • Provide a storage volume or filesystem that is attached to a workspace for the pipeline execution, using one of the following approaches:

    • Specify a volume claim template that creates a persistent volume claim
    • Specify a persistent volume claim
  • Create a PipelineRun object to instantiate and invoke the pipeline.
  • Add triggers to capture events in the source repository.

This section uses the pipelines-tutorial example to demonstrate the preceding tasks. The example uses a simple application which consists of:

  • A front-end interface, pipelines-vote-ui, with the source code in the pipelines-vote-ui Git repository.
  • A back-end interface, pipelines-vote-api, with the source code in the pipelines-vote-api Git repository.
  • The apply-manifests and update-deployment tasks in the pipelines-tutorial Git repository.

2.5.1. Prerequisites

  • You have access to an OpenShift Container Platform cluster.
  • You have installed OpenShift Pipelines using the Red Hat OpenShift Pipelines Operator listed in the OpenShift OperatorHub. Once installed, it is applicable to the entire cluster.
  • You have installed OpenShift Pipelines CLI.
  • You have forked the front-end pipelines-vote-ui and back-end pipelines-vote-api Git repositories using your GitHub ID, and have administrator access to these repositories.
  • Optional: You have cloned the pipelines-tutorial Git repository.

2.5.2. Creating a project and checking your pipeline service account

Procedure

  1. Log in to your OpenShift Container Platform cluster:

    $ oc login -u <login> -p <password> https://openshift.example.com:6443
  2. Create a project for the sample application. For this example workflow, create the pipelines-tutorial project:

    $ oc new-project pipelines-tutorial
    Note

    If you create a project with a different name, be sure to update the resource URLs used in the example with your project name.

  3. View the pipeline service account:

    Red Hat OpenShift Pipelines Operator adds and configures a service account named pipeline that has sufficient permissions to build and push an image. This service account is used by the PipelineRun object.

    $ oc get serviceaccount pipeline

2.5.3. Creating pipeline tasks

Procedure

  1. Install the apply-manifests and update-deployment task resources from the pipelines-tutorial repository, which contains a list of reusable tasks for pipelines:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/01_pipeline/01_apply_manifest_task.yaml
    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/01_pipeline/02_update_deployment_task.yaml
  2. Use the tkn task list command to list the tasks you created:

    $ tkn task list

    The output verifies that the apply-manifests and update-deployment task resources were created:

    NAME                DESCRIPTION   AGE
    apply-manifests                   1 minute ago
    update-deployment                 48 seconds ago
  3. Use the tkn clustertasks list command to list the Operator-installed additional cluster tasks such as buildah and s2i-python:

    Note

    To use the buildah cluster task in a restricted environment, you must ensure that the Dockerfile uses an internal image stream as the base image.

    $ tkn clustertasks list

    The output lists the Operator-installed ClusterTask resources:

    NAME                       DESCRIPTION   AGE
    buildah                                  1 day ago
    git-clone                                1 day ago
    s2i-python                               1 day ago
    tkn                                      1 day ago

2.5.4. Assembling a pipeline

A pipeline represents a CI/CD flow and is defined by the tasks to be executed. It is designed to be generic and reusable in multiple applications and environments.

A pipeline specifies how the tasks interact with each other and their order of execution using the from and runAfter parameters. It uses the workspaces field to specify one or more volumes that each task in the pipeline requires during execution.

In this section, you will create a pipeline that takes the source code of the application from GitHub, and then builds and deploys it on OpenShift Container Platform.

The pipeline performs the following tasks for the back-end application pipelines-vote-api and front-end application pipelines-vote-ui:

  • Clones the source code of the application from the Git repository by referring to the git-url and git-revision parameters.
  • Builds the container image using the buildah cluster task.
  • Pushes the image to the internal image registry by referring to the image parameter.
  • Deploys the new image on OpenShift Container Platform by using the apply-manifests and update-deployment tasks.

Procedure

  1. Copy the contents of the following sample pipeline YAML file and save it:

    apiVersion: tekton.dev/v1beta1
    kind: Pipeline
    metadata:
      name: build-and-deploy
    spec:
      workspaces:
      - name: shared-workspace
      params:
      - name: deployment-name
        type: string
        description: name of the deployment to be patched
      - name: git-url
        type: string
        description: url of the git repo for the code of deployment
      - name: git-revision
        type: string
        description: revision to be used from repo of the code for deployment
        default: "pipelines-1.4"
      - name: IMAGE
        type: string
        description: image to be built from the code
      tasks:
      - name: fetch-repository
        taskRef:
          name: git-clone
          kind: ClusterTask
        workspaces:
        - name: output
          workspace: shared-workspace
        params:
        - name: url
          value: $(params.git-url)
        - name: subdirectory
          value: ""
        - name: deleteExisting
          value: "true"
        - name: revision
          value: $(params.git-revision)
      - name: build-image
        taskRef:
          name: buildah
          kind: ClusterTask
        params:
        - name: IMAGE
          value: $(params.IMAGE)
        workspaces:
        - name: source
          workspace: shared-workspace
        runAfter:
        - fetch-repository
      - name: apply-manifests
        taskRef:
          name: apply-manifests
        workspaces:
        - name: source
          workspace: shared-workspace
        runAfter:
        - build-image
      - name: update-deployment
        taskRef:
          name: update-deployment
        params:
        - name: deployment
          value: $(params.deployment-name)
        - name: IMAGE
          value: $(params.IMAGE)
        runAfter:
        - apply-manifests

    The pipeline definition abstracts away the specifics of the Git source repository and image registries. These details are added as params when a pipeline is triggered and executed.

  2. Create the pipeline:

    $ oc create -f <pipeline-yaml-file-name.yaml>

    Alternatively, you can also execute the YAML file directly from the Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/01_pipeline/04_pipeline.yaml
  3. Use the tkn pipeline list command to verify that the pipeline is added to the application:

    $ tkn pipeline list

    The output verifies that the build-and-deploy pipeline was created:

    NAME               AGE            LAST RUN   STARTED   DURATION   STATUS
    build-and-deploy   1 minute ago   ---        ---       ---        ---

2.5.5. Mirroring images to run pipelines in a restricted environment

To run OpenShift Pipelines in a disconnected cluster or a cluster provisioned in a restricted environment, ensure that either the Samples Operator is configured for a restricted network, or a cluster administrator has created a cluster with a mirrored registry.

The following procedure uses the pipelines-tutorial example to create a pipeline for an application in a restricted environment using a cluster with a mirrored registry. To ensure that the pipelines-tutorial example works in a restricted environment, you must mirror the respective builder images from the mirror registry for the front-end interface, pipelines-vote-ui; back-end interface, pipelines-vote-api; and the cli.

Procedure

  1. Mirror the builder image from the mirror registry for the front-end interface, pipelines-vote-ui.

    1. Verify that the required images tag is not imported:

      $ oc describe imagestream python -n openshift

      Example output

      Name:			python
      Namespace:		openshift
      [...]
      
      3.8-ubi8 (latest)
        tagged from registry.redhat.io/ubi8/python-38:latest
          prefer registry pullthrough when referencing this tag
      
        Build and run Python 3.8 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.
        Tags: builder, python
        Supports: python:3.8, python
        Example Repo: https://github.com/sclorg/django-ex.git
      
      [...]

    2. Mirror the supported image tag to the private registry:

      $ oc image mirror registry.redhat.io/ubi8/python-38:latest <mirror-registry>:<port>/ubi8/python-38
    3. Import the image:

      $ oc tag <mirror-registry>:<port>/ubi8/python-38 python:latest --scheduled -n openshift

      You must periodically re-import the image. The --scheduled flag enables automatic re-import of the image.

    4. Verify that the images with the given tag have been imported:

      $ oc describe imagestream python -n openshift

      Example output

      Name:			python
      Namespace:		openshift
      [...]
      
      latest
        updates automatically from registry <mirror-registry>:<port>/ubi8/python-38
      
        * <mirror-registry>:<port>/ubi8/python-38@sha256:3ee3c2e70251e75bfeac25c0c33356add9cc4abcbc9c51d858f39e4dc29c5f58
      
      [...]

  2. Mirror the builder image from the mirror registry for the back-end interface, pipelines-vote-api.

    1. Verify that the required images tag is not imported:

      $ oc describe imagestream golang -n openshift

      Example output

      Name:			golang
      Namespace:		openshift
      [...]
      
      1.14.7-ubi8 (latest)
        tagged from registry.redhat.io/ubi8/go-toolset:1.14.7
          prefer registry pullthrough when referencing this tag
      
        Build and run Go applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/golang-container/blob/master/README.md.
        Tags: builder, golang, go
        Supports: golang
        Example Repo: https://github.com/sclorg/golang-ex.git
      
      [...]

    2. Mirror the supported image tag to the private registry:

      $ oc image mirror registry.redhat.io/ubi8/go-toolset:1.14.7 <mirror-registry>:<port>/ubi8/go-toolset
    3. Import the image:

      $ oc tag <mirror-registry>:<port>/ubi8/go-toolset golang:latest --scheduled -n openshift

      You must periodically re-import the image. The --scheduled flag enables automatic re-import of the image.

    4. Verify that the images with the given tag have been imported:

      $ oc describe imagestream golang -n openshift

      Example output

      Name:			golang
      Namespace:		openshift
      [...]
      
      latest
        updates automatically from registry <mirror-registry>:<port>/ubi8/go-toolset
      
        * <mirror-registry>:<port>/ubi8/go-toolset@sha256:59a74d581df3a2bd63ab55f7ac106677694bf612a1fe9e7e3e1487f55c421b37
      
      [...]

  3. Mirror the builder image from the mirror registry for the cli.

    1. Verify that the required images tag is not imported:

      $ oc describe imagestream cli -n openshift

      Example output

      Name:                   cli
      Namespace:              openshift
      [...]
      
      latest
        updates automatically from registry quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551
      
        * quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551
      
      [...]

    2. Mirror the supported image tag to the private registry:

      $ oc image mirror quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551 <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev:latest
    3. Import the image:

      $ oc tag <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev cli:latest --scheduled -n openshift

      You must periodically re-import the image. The --scheduled flag enables automatic re-import of the image.

    4. Verify that the images with the given tag have been imported:

      $ oc describe imagestream cli -n openshift

      Example output

      Name:                   cli
      Namespace:              openshift
      [...]
      
      latest
        updates automatically from registry <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev
      
        * <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551
      
      [...]

2.5.6. Running a pipeline

A PipelineRun resource starts a pipeline and ties it to the Git and image resources that should be used for the specific invocation. It automatically creates and starts the TaskRun resources for each task in the pipeline.

Procedure

  1. Start the pipeline for the back-end application:

    $ tkn pipeline start build-and-deploy \
        -w name=shared-workspace,volumeClaimTemplateFile=https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/01_pipeline/03_persistent_volume_claim.yaml \
        -p deployment-name=pipelines-vote-api \
        -p git-url=https://github.com/openshift/pipelines-vote-api.git \
        -p IMAGE=image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/pipelines-vote-api

    The previous command uses a volume claim template, which creates a persistent volume claim for the pipeline execution.

  2. To track the progress of the pipeline run, enter the following command::

    $ tkn pipelinerun logs <pipelinerun_id> -f

    The <pipelinerun_id> in the above command is the ID for the PipelineRun that was returned in the output of the previous command.

  3. Start the pipeline for the front-end application:

    $ tkn pipeline start build-and-deploy \
        -w name=shared-workspace,volumeClaimTemplateFile=https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/01_pipeline/03_persistent_volume_claim.yaml \
        -p deployment-name=pipelines-vote-ui \
        -p git-url=https://github.com/openshift/pipelines-vote-ui.git \
        -p IMAGE=image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/pipelines-vote-ui
  4. To track the progress of the pipeline run, enter the following command:

    $ tkn pipelinerun logs <pipelinerun_id> -f

    The <pipelinerun_id> in the above command is the ID for the PipelineRun that was returned in the output of the previous command.

  5. After a few minutes, use tkn pipelinerun list command to verify that the pipeline ran successfully by listing all the pipeline runs:

    $ tkn pipelinerun list

    The output lists the pipeline runs:

     NAME                         STARTED      DURATION     STATUS
     build-and-deploy-run-xy7rw   1 hour ago   2 minutes    Succeeded
     build-and-deploy-run-z2rz8   1 hour ago   19 minutes   Succeeded
  6. Get the application route:

    $ oc get route pipelines-vote-ui --template='http://{{.spec.host}}'

    Note the output of the previous command. You can access the application using this route.

  7. To rerun the last pipeline run, using the pipeline resources and service account of the previous pipeline, run:

    $ tkn pipeline start build-and-deploy --last

2.5.7. Adding triggers to a pipeline

Triggers enable pipelines to respond to external GitHub events, such as push events and pull requests. After you assemble and start a pipeline for the application, add the TriggerBinding, TriggerTemplate, Trigger, and EventListener resources to capture the GitHub events.

Procedure

  1. Copy the content of the following sample TriggerBinding YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: TriggerBinding
    metadata:
      name: vote-app
    spec:
      params:
      - name: git-repo-url
        value: $(body.repository.url)
      - name: git-repo-name
        value: $(body.repository.name)
      - name: git-revision
        value: $(body.head_commit.id)
  2. Create the TriggerBinding resource:

    $ oc create -f <triggerbinding-yaml-file-name.yaml>

    Alternatively, you can create the TriggerBinding resource directly from the pipelines-tutorial Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/03_triggers/01_binding.yaml
  3. Copy the content of the following sample TriggerTemplate YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: TriggerTemplate
    metadata:
      name: vote-app
    spec:
      params:
      - name: git-repo-url
        description: The git repository url
      - name: git-revision
        description: The git revision
        default: pipelines-1.4
      - name: git-repo-name
        description: The name of the deployment to be created / patched
    
      resourcetemplates:
      - apiVersion: tekton.dev/v1beta1
        kind: PipelineRun
        metadata:
          generateName: build-deploy-$(tt.params.git-repo-name)-
        spec:
          serviceAccountName: pipeline
          pipelineRef:
            name: build-and-deploy
          params:
          - name: deployment-name
            value: $(tt.params.git-repo-name)
          - name: git-url
            value: $(tt.params.git-repo-url)
          - name: git-revision
            value: $(tt.params.git-revision)
          - name: IMAGE
            value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/$(tt.params.git-repo-name)
          workspaces:
          - name: shared-workspace
            volumeClaimTemplate:
              spec:
                accessModes:
                  - ReadWriteOnce
                resources:
                  requests:
                    storage: 500Mi

    The template specifies a volume claim template to create a persistent volume claim for defining the storage volume for the workspace. Therefore, you do not need to create a persistent volume claim to provide data storage.

  4. Create the TriggerTemplate resource:

    $ oc create -f <triggertemplate-yaml-file-name.yaml>

    Alternatively, you can create the TriggerTemplate resource directly from the pipelines-tutorial Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/03_triggers/02_template.yaml
  5. Copy the contents of the following sample Trigger YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: Trigger
    metadata:
      name: vote-trigger
    spec:
      serviceAccountName: pipeline
      bindings:
        - ref: vote-app
      template:
        ref: vote-app
  6. Create the Trigger resource:

    $ oc create -f <trigger-yaml-file-name.yaml>

    Alternatively, you can create the Trigger resource directly from the pipelines-tutorial Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/03_triggers/03_trigger.yaml
  7. Copy the contents of the following sample EventListener YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: EventListener
    metadata:
      name: vote-app
    spec:
      serviceAccountName: pipeline
      triggers:
        - triggerRef: vote-trigger

    Alternatively, if you have not defined a trigger custom resource, add the binding and template spec to the EventListener YAML file, instead of referring to the name of the trigger:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: EventListener
    metadata:
      name: vote-app
    spec:
      serviceAccountName: pipeline
      triggers:
      - bindings:
        - ref: vote-app
        template:
          ref: vote-app
  8. Create the EventListener resource by performing the following steps:

    • To create an EventListener resource using a secure HTTPS connection:

      1. Add a label to enable the secure HTTPS connection to the Eventlistener resource:

        $ oc label namespace <ns-name> operator.tekton.dev/enable-annotation=enabled
      2. Create the EventListener resource:

        $ oc create -f <eventlistener-yaml-file-name.yaml>

        Alternatively, you can create the EvenListener resource directly from the pipelines-tutorial Git repository:

        $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.4/03_triggers/04_event_listener.yaml
      3. Create a route with the re-encrypt TLS termination:

        $ oc create route reencrypt --service=<svc-name> --cert=tls.crt --key=tls.key --ca-cert=ca.crt --hostname=<hostname>

        Alternatively, you can create a re-encrypt TLS termination YAML file to create a secured route.

        Example Re-encrypt TLS Termination YAML of the Secured Route

        apiVersion: v1
        kind: Route
        metadata:
          name: route-passthrough-secured 1
        spec:
          host: <hostname>
          to:
            kind: Service
            name: frontend 2
          tls:
            termination: reencrypt         3
            key: [as in edge termination]
            certificate: [as in edge termination]
            caCertificate: [as in edge termination]
            destinationCACertificate: |-   4
              -----BEGIN CERTIFICATE-----
              [...]
              -----END CERTIFICATE-----

        1 2
        The name of the object, which is limited to 63 characters.
        3
        The termination field is set to reencrypt. This is the only required tls field.
        4
        Required for re-encryption. destinationCACertificate specifies a CA certificate to validate the endpoint certificate, securing the connection from the router to the destination pods. If the service is using a service signing certificate, or the administrator has specified a default CA certificate for the router and the service has a certificate signed by that CA, this field can be omitted.

        See oc create route reencrypt --help for more options.

    • To create an EventListener resource using an insecure HTTP connection:

      1. Create the EventListener resource.
      2. Expose the EventListener service as an OpenShift Container Platform route to make it publicly accessible:

        $ oc expose svc el-vote-app

2.5.8. Creating webhooks

Webhooks are HTTP POST messages that are received by the event listeners whenever a configured event occurs in your repository. The event payload is then mapped to trigger bindings, and processed by trigger templates. The trigger templates eventually start one or more pipeline runs, leading to the creation and deployment of Kubernetes resources.

In this section, you will configure a webhook URL on your forked Git repositories pipelines-vote-ui and pipelines-vote-api. This URL points to the publicly accessible EventListener service route.

Note

Adding webhooks requires administrative privileges to the repository. If you do not have administrative access to your repository, contact your system administrator for adding webhooks.

Procedure

  1. Get the webhook URL:

    • For a secure HTTPS connection:

      $ echo "URL: $(oc  get route el-vote-app --template='https://{{.spec.host}}')"
    • For an HTTP (insecure) connection:

      $ echo "URL: $(oc  get route el-vote-app --template='http://{{.spec.host}}')"

      Note the URL obtained in the output.

  2. Configure webhooks manually on the front-end repository:

    1. Open the front-end Git repository pipelines-vote-ui in your browser.
    2. Click SettingsWebhooksAdd Webhook
    3. On the Webhooks/Add Webhook page:

      1. Enter the webhook URL from step 1 in Payload URL field
      2. Select application/json for the Content type
      3. Specify the secret in the Secret field
      4. Ensure that the Just the push event is selected
      5. Select Active
      6. Click Add Webhook
  3. Repeat step 2 for the back-end repository pipelines-vote-api.

2.5.9. Triggering a pipeline run

Whenever a push event occurs in the Git repository, the configured webhook sends an event payload to the publicly exposed EventListener service route. The EventListener service of the application processes the payload, and passes it to the relevant TriggerBinding and TriggerTemplate resource pairs. The TriggerBinding resource extracts the parameters, and the TriggerTemplate resource uses these parameters and specifies the way the resources must be created. This may rebuild and redeploy the application.

In this section, you push an empty commit to the front-end pipelines-vote-ui repository, which then triggers the pipeline run.

Procedure

  1. From the terminal, clone your forked Git repository pipelines-vote-ui:

    $ git clone git@github.com:<your GitHub ID>/pipelines-vote-ui.git -b pipelines-1.4
  2. Push an empty commit:

    $ git commit -m "empty-commit" --allow-empty && git push origin pipelines-1.4
  3. Check if the pipeline run was triggered:

    $ tkn pipelinerun list

    Notice that a new pipeline run was initiated.

2.5.10. Additional resources

2.6. Working with Red Hat OpenShift Pipelines using the Developer perspective

You can use the Developer perspective of the OpenShift Container Platform web console to create CI/CD pipelines for your software delivery process.

In the Developer perspective:

  • Use the AddPipelinePipeline Builder option to create customized pipelines for your application.
  • Use the AddFrom Git option to create pipelines using operator-installed pipeline templates and resources while creating an application on OpenShift Container Platform.

After you create the pipelines for your application, you can view and visually interact with the deployed pipelines in the Pipelines view. You can also use the Topology view to interact with the pipelines created using the From Git option. You need to apply custom labels to a pipeline created using the Pipeline Builder to see it in the Topology view.

Prerequisites

  • You have access to an OpenShift Container Platform cluster and have switched to the Developer perspective in the web console.
  • You have the OpenShift Pipelines Operator installed in your cluster.
  • You are a cluster administrator or a user with create and edit permissions.
  • You have created a project.

2.6.1. Constructing Pipelines using the Pipeline Builder

In the Developer perspective of the console, you can use the +AddPipelinePipeline builder option to:

  • Configure pipelines using either the Pipeline builder or the YAML view.
  • Construct a pipeline flow using existing tasks and cluster tasks. When you install the OpenShift Pipelines Operator, it adds reusable pipeline cluster tasks to your cluster.
  • Specify the type of resources required for the pipeline run, and if required, add additional parameters to the pipeline.
  • Reference these pipeline resources in each of the tasks in the pipeline as input and output resources.
  • If required, reference any additional parameters added to the pipeline in the task. The parameters for a task are prepopulated based on the specifications of the task.
  • Use the Operator-installed, reusable snippets and samples to create detailed pipelines.

Procedure

  1. In the +Add view of the Developer perspective, click the Pipeline tile to see the Pipeline builder page.
  2. Configure the pipeline using either the Pipeline builder view or the YAML view.

    Note

    The Pipeline builder view supports a limited number of fields whereas the YAML view supports all available fields. Optionally, you can also use the Operator-installed, reusable snippets and samples to create detailed Pipelines.

    Figure 2.1. YAML view

    op pipeline yaml

    To configure your pipeline using the Pipeline Builder:

    1. Enter a unique name for the pipeline.
    2. Select a task from the Select Task list to add a task to the pipeline. This example uses the s2i-nodejs task.

      • To add sequential tasks to the pipeline, click the plus icon to the right or left of the task, and from the Select Task list, select the task you want to add to the pipeline. For this example, use the plus icon to the right of the s2i-nodejs task to add an openshift-client task.
      • To add a parallel task to the existing task, click the plus icon displayed next to the task, and from the Select Task list, select the parallel task you want to add to the pipeline.

        Figure 2.2. Pipeline Builder

        op pipeline builder
    3. Click Add Resources to specify the name and type of resources that the pipeline run will use. These resources are then used by the tasks in the pipeline as inputs and outputs. For this example:

      1. Add an input resource. In the Name field, enter Source, and then from the Resource Type drop-down list, select Git.
      2. Add an output resource. In the Name field, enter Img, and then from the Resource Type drop-down list, select Image.
    4. Optional: The Parameters for a task are prepopulated based on the specifications of the task. If required, use the Add Parameters link to add additional parameters.
    5. A Missing Resources warning is displayed on a task if the resources for the task are not specified. Click the s2i-nodejs task to see the side panel with details for the task.

      Figure 2.3. Tasks details in Pipelines Builder

      op pipeline builder task details
    6. In the task side panel, specify the resources and parameters for the s2i-nodejs task:

      1. In the Input ResourcesSource section, the Select Resources drop-down list displays the resources that you added to the pipeline. For this example, select Source.
      2. In the Output ResourcesImage section, click the Select Resources list, and select Img.
      3. If required, in the Parameters section, add more parameters to the default ones, by using the $(params.<param-name>) syntax.
      4. Similarly, add an input resource for the openshift-client task.
  3. Click Create to create and view the pipeline in the Pipeline Details page.
  4. Click the Actions drop-down menu, and then click Start to start the Pipeline.

2.6.2. Creating applications with OpenShift Pipelines

To create pipelines along with applications, use the From Git option in the Add view of the Developer perspective. For more information, see Creating applications using the Developer perspective.

2.6.3. Interacting with pipelines using the Developer perspective

The Pipelines view in the Developer perspective lists all the pipelines in a project, along with the following details:

  • The namespace in which the pipeline was created
  • The last pipeline run
  • The status of the tasks in the pipeline run
  • The status of the pipeline run
  • The creation time of the last pipeline run

Procedure

  1. In the Pipelines view of the Developer perspective, select a project from the Project drop-down list to see the pipelines in that project.
  2. Click the required pipeline to see the Pipeline details page. By default, the Details tab opens and provides a visual representation of all the serial and parallel tasks in the pipeline. The tasks are also listed in the lower right portion of the page. You can click the listed Tasks to view the task details.

    Figure 2.4. Pipeline details

    Pipeline details
  3. Optionally, in the Pipeline details page:

    • Click the Metrics tab to see the following information about pipelines:

      • Pipeline Success Ratio
      • Number of Pipeline Runs
      • Pipeline Run Duration
      • Task Run Duration

        You can use this information to improve the pipeline workflow and eliminate issues early in the pipeline lifecycle.

    • Click the YAML tab to edit the YAML file for the pipeline.
    • Click the Pipeline Runs tab to see the completed, running, or failed runs for the pipeline.

      Note

      The Details section of the Pipeline Run Details page displays a Log Snippet of the failed pipeline run. Log Snippet provides a general error message and a snippet of the log. A link to the Logs section provides quick access to the details about the failed run. The Log Snippet is also displayed in the Details section of the Task Run Details page.

      You can use the Options menu kebab to stop a running pipeline, to rerun a pipeline using the same parameters and resources as that of the previous pipeline execution, or to delete a pipeline run.

    • Click the Parameters tab to see the parameters defined in the pipeline. You can also add or edit additional parameters, as required.
    • Click the Resources tab to see the resources defined in the pipeline. You can also add or edit additional resources, as required.

2.6.4. Starting pipelines

After you create a pipeline, you need to start it to execute the included tasks in the defined sequence. You can start a pipeline from the Pipelines view, the Pipeline Details page, or the Topology view.

Procedure

To start a pipeline using the Pipelines view:

  1. In the Pipelines view of the Developer perspective, click the Options kebab menu adjoining a pipeline, and select Start.
  2. The Start Pipeline dialog box displays the Git Resources and the Image Resources based on the pipeline definition.

    Note

    For pipelines created using the From Git option, the Start Pipeline dialog box also displays an APP_NAME field in the Parameters section, and all the fields in the dialog box are prepopulated by the pipeline template.

    1. If you have resources in your namespace, the Git Resources and the Image Resources fields are prepopulated with those resources. If required, use the drop-downs to select or create the required resources and customize the pipeline run instance.
  3. Optional: Modify the Advanced Options to add the credentials that authenticate the specified private Git server or the image registry.

    1. Under Advanced Options, click Show Credentials Options and select Add Secret.
    2. In the Create Source Secret section, specify the following:

      1. A unique Secret Name for the secret.
      2. In the Designated provider to be authenticated section, specify the provider to be authenticated in the Access to field, and the base Server URL.
      3. Select the Authentication Type and provide the credentials:

        • For the Authentication Type Image Registry Credentials, specify the Registry Server Address that you want to authenticate, and provide your credentials in the Username, Password, and Email fields.

          Select Add Credentials if you want to specify an additional Registry Server Address.

        • For the Authentication Type Basic Authentication, specify the values for the UserName and Password or Token fields.
        • For the Authentication Type SSH Keys, specify the value of the SSH Private Key field.
      4. Select the check mark to add the secret.

    You can add multiple secrets based upon the number of resources in your pipeline.

  4. Click Start to start the pipeline.
  5. The Pipeline Run Details page displays the pipeline being executed. After the pipeline starts, the tasks and steps within each task are executed. You can:

    • Hover over the tasks to see the time taken to execute each step.
    • Click on a task to see the logs for each step in the task.
    • Click the Logs tab to see the logs relating to the execution sequence of the tasks. You can also expand the pane and download the logs individually or in bulk, by using the relevant button.
    • Click the Events tab to see the stream of events generated by a pipeline run.

      You can use the Task Runs, Logs, and Events tabs to assist in debugging a failed pipeline run or a failed task run.

      Figure 2.5. Pipeline run details

      Pipeline run details
  6. For pipelines created using the From Git option, you can use the Topology view to interact with pipelines after you start them:

    Note

    To see the pipelines created using the Pipeline Builder in the Topology view, customize the pipeline labels to link the pipeline with the application workload.

    1. On the left navigation panel, click Topology, and click on the application to see the pipeline runs listed in the side panel.
    2. In the Pipeline Runs section, click Start Last Run to start a new pipeline run with the same parameters and resources as the previous one. This option is disabled if a pipeline run has not been initiated.

      Figure 2.6. Pipelines in Topology view

      Pipelines in Topology view
    3. In the Topology page, hover to the left of the application to see the status of the pipeline run for the application.

      Note

      The side panel of the application node in the Topology page displays a Log Snippet when a pipeline run fails on a specific task run. You can view the Log Snippet in the Pipeline Runs section, under the Resources tab. Log Snippet provides a general error message and a snippet of the log. A link to the Logs section provides quick access to the details about the failed run.

2.6.5. Editing Pipelines

You can edit the Pipelines in your cluster using the Developer perspective of the web console:

Procedure

  1. In the Pipelines view of the Developer perspective, select the Pipeline you want to edit to see the details of the Pipeline. In the Pipeline Details page, click Actions and select Edit Pipeline.
  2. In the Pipeline Builder page:

    • You can add additional Tasks, parameters, or resources to the Pipeline.
    • You can click the Task you want to modify to see the Task details in the side panel and modify the required Task details, such as the display name, parameters and resources.
    • Alternatively, to delete the Task, click the Task, and in the side panel, click Actions and select Remove Task.
  3. Click Save to save the modified Pipeline.

2.6.6. Deleting Pipelines

You can delete the Pipelines in your cluster using the Developer perspective of the web console.

Procedure

  1. In the Pipelines view of the Developer perspective, click the Options kebab menu adjoining a Pipeline, and select Delete Pipeline.
  2. In the Delete Pipeline confirmation prompt, click Delete to confirm the deletion.

2.7. Reducing resource consumption of pipelines

If you use clusters in multi-tenant environments you must control the consumption of CPU, memory, and storage resources for each project and Kubernetes object. This helps prevent any one application from consuming too many resources and affecting other applications.

To define the final resource limits that are set on the resulting pods, Red Hat OpenShift Pipelines use resource quota limits and limit ranges of the project in which they are executed.

To restrict resource consumption in your project, you can:

2.7.1. Understanding resource consumption in pipelines

Each task consists of a number of required steps to be executed in a particular order defined in the steps field of the Task resource. Every task runs as a pod, and each step runs as a container within that pod.

Steps are executed one at a time. The pod that executes the task only requests enough resources to run a single container image (step) in the task at a time, and thus does not store resources for all the steps in the task.

The Resources field in the steps spec specifies the limits for resource consumption. By default, the resource requests for the CPU, memory, and ephemeral storage are set to BestEffort (zero) values or to the minimums set through limit ranges in that project.

Example configuration of resource requests and limits for a step

spec:
  steps:
  - name: <step_name>
    resources:
      requests:
        memory: 2Gi
        cpu: 600m
      limits:
        memory: 4Gi
        cpu: 900m

When the LimitRange parameter and the minimum values for container resource requests are specified in the project in which the pipeline and task runs are executed, Red Hat OpenShift Pipelines looks at all the LimitRange values in the project and uses the minimum values instead of zero.

Example configuration of limit range parameters at a project level

apiVersion: v1
kind: LimitRange
metadata:
  name: <limit_container_resource>
spec:
  limits:
  - max:
      cpu: "600m"
      memory: "2Gi"
    min:
      cpu: "200m"
      memory: "100Mi"
    default:
      cpu: "500m"
      memory: "800Mi"
    defaultRequest:
      cpu: "100m"
      memory: "100Mi"
    type: Container
...

2.7.2. Mitigating extra resource consumption in pipelines

When you have resource limits set on the containers in your pod, OpenShift Container Platform sums up the resource limits requested as all containers run simultaneously.

To consume the minimum amount of resources needed to execute one step at a time in the invoked task, Red Hat OpenShift Pipelines requests the maximum CPU, memory, and ephemeral storage as specified in the step that requires the most amount of resources. This ensures that the resource requirements of all the steps are met. Requests other than the maximum values are set to zero.

However, this behavior can lead to higher resource usage than required. If you use resource quotas, this could also lead to unschedulable pods.

For example, consider a task with two steps that uses scripts, and that does not define any resource limits and requests. The resulting pod has two init containers (one for entrypoint copy, the other for writing scripts) and two containers, one for each step.

OpenShift Container Platform uses the limit range set up for the project to compute required resource requests and limits. For this example, set the following limit range in the project:

apiVersion: v1
kind: LimitRange
metadata:
  name: mem-min-max-demo-lr
spec:
  limits:
  - max:
      memory: 1Gi
    min:
      memory: 500Mi
    type: Container

In this scenario, each init container uses a request memory of 1Gi (the max limit of the limit range), and each container uses a request memory of 500Mi. Thus, the total memory request for the pod is 2Gi.

If the same limit range is used with a task of ten steps, the final memory request is 5Gi, which is higher than what each step actually needs, that is 500Mi (since each step runs after the other).

Thus, to reduce resource consumption of resources, you can:

  • Reduce the number of steps in a given task by grouping different steps into one bigger step, using the script feature, and the same image. This reduces the minimum requested resource.
  • Distribute steps that are relatively independent of each other and can run on their own to multiple tasks instead of a single task. This lowers the number of steps in each task, making the request for each task smaller, and the scheduler can then run them when the resources are available.

2.7.3. Additional Resources

2.8. Using pods in a privileged security context

The default configuration of OpenShift Pipelines 1.3.x and later versions does not allow you to run pods with privileged security context, if the pods result from pipeline run or task run. For such pods, the default service account is pipeline, and the security context constraint (SCC) associated with the pipelines service account is pipelines-scc. The pipelines-scc SCC is similar to the anyuid SCC, but with a minor difference as defined in the YAML file for the SCC of pipelines:

Example SecurityContextConstraints object

apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
...
fsGroup:
  type: MustRunAs
...

In addition, the Buildah cluster task, shipped as part of the OpenShift Pipelines, uses vfs as the default storage driver.

2.8.1. Running pipeline run and task run pods with privileged security context

Procedure

To run a pod (resulting from pipeline run or task run) with the privileged security context, do the following modifications:

  • Configure the associated user account or service account to have an explicit SCC. You can perform the configuration using any of the following methods:

    • Execute the following OpenShift command:

      $ oc adm policy add-scc-to-user <scc-name> -z <service-account-name>
    • Alternatively, modify the YAML files for RoleBinding, and Role or ClusterRole:

      Example RoleBinding object

      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: service-account-name 1
        namespace: default
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: pipelines-scc-clusterrole 2
      subjects:
      - kind: ServiceAccount
        name: pipeline
        namespace: default

      1
      Substitute with an appropriate service account name.
      2
      Substitute with an appropriate cluster role based on the role binding you use.

      Example ClusterRole object

      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: pipelines-scc-clusterrole 1
      rules:
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot
        resources:
        - securitycontextconstraints
        verbs:
        - use

      1
      Substitute with an appropriate cluster role based on the role binding you use.
    Note

    As a best practice, create a copy of the default YAML files and make changes in the duplicate file.

  • If you do not use the vfs storage driver, configure the service account associated with the task run or the pipeline run to have a privileged SCC, and set the security context as privileged: true.

2.8.2. Additional resources