OCP 4.14: Installing SAP Data Intelligence 3 on OpenShift Container Platform 4 supported by SDI Observer Operator
Contents
- 1. Introduction
- 2. OpenShift Container Platform validation version matrix
- 3. Requirements
- 4. Installing Red Hat OpenShift Container Platform
- 4.1. Preparing the Management host
- 4.2. Installing OpenShift Container Platform
- 4.3. Post installation steps
- 4.4. Installing SDI Observer Operator
- 4.4.1. Install SDI Observer Operator
- 4.4.2. Managing SDIObserver instance
- 4.4.3. (Optional) Enabling SDI on control plane
- 4.4.4. Verifying node configuration
- 4.4.5. Deploying persistent storage provider
- 4.4.6. Configuring S3 access and bucket
- 4.4.7. Setting up a container image registry
- 4.4.8. Configuring the OpenShift cluster for SDI
- 5. Installing SDI on OpenShift
- 6. Upgrading OpenShift Container Platform
- 7. Upgrading or updating SAP Data Intelligence
- 8. Appendix
- 8.1. Uninstalling SDI
- 8.2. Quay Registry for SDI
- 8.3. Configuring OpenShift to trust container image registry
- 8.4. Configuring insecure registry
- 8.5. Running multiple SDI instances on a single OpenShift cluster
- 8.6. Installing remarshal utilities on RHEL
- 8.7. Upgrading to the next minor release from the latest asynchronous release
- 8.8. HTTP proxy configuration
- 8.9. Enabling GPU for SDI on OCP
- 9. Troubleshooting
1. Introduction
In general, the installation of SAP Data Intelligence (SDI) follows these steps:
- Install the Red Hat OpenShift Container Platform.
- Install SDI Observer Operator.
- Configure the prerequisites for SAP Data Intelligence with the SDI Observer Operator.
- Install SAP Data Intelligence on the OpenShift Container Platform.
If you're interested in previous versions or former methods of installation of SAP Data Intelligence, please refer to SAP Data Intelligence 3 on OpenShift Container Platform 4 guide.
NOTE
- OpenShift Container Storage (OCS) is referred to throughout this article under its new product name, OpenShift Data Foundation (ODF).
- OpenShift Container Platform (OCP) can be substituted by the OpenShift Kubernetes Engine (OKE). OKE is sufficient and supported to run SAP Data Intelligence.
- There are known SAP image security issues that may be revealed during a security audit. Red Hat cannot resolve them. Please open a support case with SAP regarding any of the following:
- SAP containers run as root
- SAP containers run unconfined (unrestricted by SELinux)
- SAP containers require a privileged security context
2. OpenShift Container Platform validation version matrix
The following version combinations of SDI 3.X, OpenShift Container Platform (OCP), SDI Observer Operator, RHEL or RHCOS have been validated for production environments:
SAP Data Intelligence | OpenShift Container Platform | Operating System | Infrastructure and (Storage) | Confirmed&Supported by SAP |
---|---|---|---|---|
3.3 | 4.14, 4.16 | RHCOS (nodes), RHEL 8.6+ or Fedora (Management host) | VMware vSphere (ODF 4) | supported |
3.3 | 4.14, 4.16 | RHCOS (nodes), RHEL 8.6+ or Fedora (Management host) | Bare metal ∗ (ODF 4 ¡) | supported |
+ The referenced OpenShift release is no longer supported by Red Hat!
∗ Validated on two different hardware configurations.
-
For more information on OCP on IBM Cloud™, please refer to Getting started with Red Hat OpenShift on IBM Cloud.
If using this platform, you don't need to install OpenShift and may jump directly to IBM's documentation Planning your SAP Data Intelligence deployment. You will be guided through all installation steps and find the appropriate links back to this Red Hat article. -
(Dev/PoC level) Lenovo 4 bare metal hosts setup is composed of:
-
3 schedulable control plane nodes running both ODF and SDI (Lenovo ThinkSystem SR530)
-
1 Compute node running SDI (Lenovo ThinkSystem SR530)
-
(Production level) Dell Technologies bare metal cluster is composed of:
-
1 CSAH node (Dell EMC PowerEdge R640s)
- 3 control plane nodes (Dell EMC PowerEdge R640s)
- 3 dedicated ODF nodes (Dell EMC PowerEdge R640s)
- 3 dedicated SDI nodes (Dell EMC PowerEdge R740xd)
CSI supported external Dell EMC storage options and cluster sizing options are available.
CSAH stands for Cluster System Admin Host - an equivalent of a Management host.
Please refer to the compatibility matrix for version combinations that are considered working. You can also refer to SAP Note #2871970 for more details.
3. Requirements
3.1. Hardware/VM and OS requirements
3.1.1. OpenShift cluster
Make sure to check the following official cluster requirements:
- SAP Data Intelligence in SAP's documentation:
- OpenShift 4 - Minimum resource requirements
- Additionally, if deploying OpenShift Data Foundation (aka ODF), please refer to ODF Supported configurations
- When deploying on VMware vSphere, please also consider the VMware vSphere infrastructure requirements
- When deploying NetApp Trident, please also refer to NetApp Hardware/VM and OS Requirements
3.1.1.1. Different types of nodes
There are four kinds of nodes:
- Bootstrap Node - A temporary bootstrap node is needed for the OpenShift deployment. The node can be either destroyed by the installer (using infrastructure-provisioned-installation, aka IPI) or deleted manually by the administrator. Alternatively, it can be reused as a worker node. Please refer to the Installation process (4.14) for more information.
- Master Nodes (4.14) - The control plane manages the OpenShift Container Platform cluster. The control plane can be made schedulable to enable SDI workload there as well.
- Compute Nodes (4.14) - Run the actual workload (e.g., SDI pods). They are optional on a three-node cluster (where the master nodes are schedulable).
- ODF Nodes (4.12) - Run OpenShift Data Foundation (aka ODF). The nodes can be divided into starting nodes (running both OSDs and monitors) and additional nodes (running only OSDs). It is needed only when ODF is used as the backup storage provider.
- NOTE Running in compact mode (on the control plane) is fully supported starting from ODF 4.8.
-
Management host (aka administrator's workstation or Jump host - The Management host is used, among other things, for:
- Accessing the OpenShift cluster via a configured command line client (
oc
orkubectl
). - Configuring an OpenShift cluster.
- Running Software Lifecycle Container Bridge (SLC Bridge).
- Accessing the OpenShift cluster via a configured command line client (
The hardware/software requirements for the Management host can be:
- OS: Red Hat Enterprise Linux 8.1+, RHEL 7.6+ or Fedora 30+
- Diskspace: 20GiB
3.1.1.2. [Not Supported] Disconnected and air-gapped environments
Disconnected and air-gapped environments installations are validated for the other method of SAP Data Intelligence installation, which is not in the scope of guidance. Please refer to the validated installation method of SAP Data Intelligence 3 on OpenShift Container Platform 4 for deploying in disconnected or air-gapped environments.
3.1.1.3. Minimum hardware requirements
The table below lists the minimum requirements and the minimum number of instances for each node type for the latest validated SDI and OpenShift 4.X releases. This is sufficient for a PoC (Proof of Concept) environments.
Type | Count | Operating System | vCPU ⑃ | RAM (GB) | Storage (GB) | AWS Instance Type |
---|---|---|---|---|---|---|
Bootstrap | 1 | RHCOS | 4 | 16 | 120 | m4.xlarge |
Master | 3 | RHCOS | 4 | 16 | 120 | m4.xlarge |
Compute | 3+ | RHCOS or RHEL 8.6, 8.7 or 8.8 | 8 | 32 | 120 | m4.2xlarge |
On a three-node cluster, it would look like this:
Type | Count | Operating System | vCPU ⑃ | RAM (GB) | Storage (GB) | AWS Instance Type |
---|---|---|---|---|---|---|
Bootstrap | 1 | RHCOS | 4 | 16 | 120 | m4.xlarge |
Master/Compute | 3 | RHCOS | 10 | 40 | 120 | m4.xlarge |
If using ODF in internal mode, at least three additional (starting) nodes are recommended. Alternatively, the Compute nodes outlined above can also run ⑂ ODF pods. In that case, the hardware specifications need to be extended accordingly. The following table lists the minimum requirements for each additional node:
Type | Count | Operating System | vCPU ⑃ | RAM (GB) | Storage (GB) | AWS Instance Type |
---|---|---|---|---|---|---|
ODF starting (OSD+MON) | 3 | RHCOS | 10 | 24 | 120 + 2048 ♢ | m5.4xlarge |
3.1.1.4. Minimum production hardware requirements
The minimum production requirements for production systems for the latest validated SDI and OpenShift 4 are the following:
Type | Count | Operating System | vCPU ⑃ | RAM (GB) | Storage (GB) | AWS Instance Type |
---|---|---|---|---|---|---|
Bootstrap | 1 | RHCOS | 4 | 16 | 120 | m4.xlarge |
Master | 3+ | RHCOS | 8 | 16 | 120 | c5.xlarge |
Compute | 3+ | RHCOS or RHEL 8.6, 8.7 or 8.8 | 16 | 64 | 120 | m4.4xlarge |
On a three-node cluster, it would look like this:
Type | Count | Operating System | vCPU ⑃ | RAM (GB) | Storage (GB) | AWS Instance Type |
---|---|---|---|---|---|---|
Bootstrap | 1 | RHCOS | 4 | 16 | 120 | m4.xlarge |
Master/Compute | 3 | RHCOS | 22 | 72 | 120 | c5.9xlarge |
If using ODF 4 in internal mode, at least 3 additional (starting) nodes are recommended. Alternatively, the Compute nodes outlined above can also run ODF ⑂ pods. In that case, the hardware specifications need to be extended accordingly. The following table lists the minimum requirements for each additional node:
Type | Count | Operating System | vCPU ⑃ | RAM (GB) | Storage (GB) | AWS Instance Type |
---|---|---|---|---|---|---|
ODF starting (OSD+MON) | 3 | RHCOS | 20 | 49 | 120 + 6×2048 ♢ | c5a.8xlarge |
♢ Please refer to ODF Platform Requirements (4.12).
⑂ Running in a compact mode (on the control plane) is fully supported starting with ODF 4.8.
⑃ 1 physical core provides 2 vCPUs when hyper-threading is enabled. One physical core provides one vCPU when hyper-threading is not enabled.
3.2. Software requirements
3.2.1. Compatibility matrix
Later versions of SAP Data Intelligence support newer versions of Kubernetes and the OpenShift Container Platform or OpenShift Kubernetes Engine. Even if not listed in the OpenShift validation version matrix above, the following version combinations are considered fully working and supported:
SAP Data Intelligence | OpenShift Container Platform ² | Worker Node | Management host | Infrastructure | Storage | Object Storage |
---|---|---|---|---|---|---|
3.3 | 4.14, 4.16 | RHCOS | RHEL 8.1 or newer | Cloud ❄, VMware vSphere, Bare metal | ODF 4, NetApp Trident 20.04 or newer, vSphere volumes ♣, NFS ♣ | ODF ¡, NetApp StorageGRID 11.4 or newer |
² OpenShift Kubernetes Engine (OKE) is a viable and supported substitute for OpenShift Container Platform (OCP).
❄ Cloud means any cloud provider supported by the OpenShift Container Platform. For a complete list of tested and supported infrastructure platforms, please refer to OpenShift Container Platform 4.x Tested Integrations. The persistent storage in this case must be provided by the cloud provider. Please refer to Understanding persistent storage (4.14) for a complete list of supported storage providers.
♣ This persistent storage provider does not offer the supported object storage service required by SDI's checkpoint store and therefore is suitable only for SAP Data Intelligence development and PoC clusters. It needs to be complemented by an object storage solution for full SDI functionality.
¡ For full functionality (including SDI backup and restore), ODF 4.6.4 or newer is required. Alternatively, ODF external mode can be used while utilizing RGW for SDI backup and restore (checkpoint store).
Unless stated otherwise, the compatibility of a listed SDI version covers all its patch releases as well.
3.2.2. Persistent volumes
Persistent storage is needed for SDI. It is required to use storage that can be created dynamically. You can find more information in the Understanding persistent storage (4.14) document.
3.2.3. Container image registry
The SDI installation requires a secured image registry where images are first mirrored from an SAP registry and then delivered to the OpenShift cluster nodes. The integrated OpenShift container registry (4.14) is NOT appropriate for this purpose. Neither is the AWS ECR Registry. For now, another image registry needs to be set up instead.
The requirements listed here are a subset of the official requirements listed in the Container Registry (3.3).
The word secured in this context means that the communication is encrypted using TLS. Ideally, with certificates signed by a trusted certificate authority. If the registry is also exposed publicly, it must require authentication and authorization in order to pull SAP images.
3.2.3.1. Validated registries
-
(Recommended) Red Hat Quay 3.6 or higher is compatible with SAP Data Intelligence images and is supported for this purpose. The Quay registry can run either on OpenShift cluster itself, another OpenShift cluster or standalone. For more information, please see the Quay Registry for SAP DI.
-
(Deprecated) SDI Registry is a community-supported container image registry. It is currently not supported by the SAP Data Intelligence Observer Operator. You need to refer to SAP Data Intelligence 3 on OpenShift Container Platform 4 for deploying the community-supported container image registry.
When finished, you should have an external image registry up and running. We will use the URL local.image.registry:5000
as an example. You can verify its readiness with the following command:
# curl -k https://local.image.registry:5000/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}
3.2.4. Checkpoint store enablement
In order to enable SAP Vora Database streaming tables, checkpoint stores need to be enabled. The store is an object stored on a particular storage backend. Several back-end types are supported by the SDI installer that cover most of the storage cloud providers.
The enablement is strongly recommended for production clusters. Clusters with this feature disabled are suitable only for test, development or PoC use cases.
Make sure to create the desired bucket before the SDI installation. If the checkpoint store will reside in a directory on a bucket, the directory needs to exist as well.
3.2.5. SDI Observer Operator
It is an operator for monitoring the SAP Data Intelligence (SDI) namespace and modifying objects there that enable the running of SDI on top of OpenShift. The observer shall be run in a dedicated namespace. It must be deployed before the SDI installation is started. SDI Observer Operator section will guide you through the process of deployment.
4. Installing Red Hat OpenShift Container Platform
4.1. Preparing the Management host
NOTE The following has been tested on RHEL 8.4. The steps will be similar for other RPM based Linux distributions. Recommended are RHEL 7.7+, Fedora 30+ and CentOS 7+.
4.1.1. Preparing the connected Management host
- Subscribe to the Management host for at least the following repositories:
# OCP_RELEASE=4.14
# sudo subscription-manager repos \
--enable=rhel-8-for-x86_64-appstream-rpms \
--enable=rhel-8-for-x86_64-baseos-rpms \
--enable=rhocp-${OCP_RELEASE:-4.14}-for-rhel-8-x86_64-rpms
- Install
jq
binary. This installation guide has been tested with jq 1.6.
-
On RHEL 8, make sure the
rhocp-4.14-for-rhel-8-x86_64-rpms
repository or newer is enabled and install it from there:# dnf install jq-1.6
-
On earlier releases or other distributions, download the binary from upstream:
# sudo curl -L -O /usr/local/bin/jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 # sudo chmod a+x /usr/local/bin/jq
- Download and install OpenShift client binaries.
# sudo dnf install -y openshift-clients
Install jq-1.6
and openshift-clients
from your local RPM repository.
4.2. Installing OpenShift Container Platform
Install OpenShift Container Platform on your desired cluster hosts. Follow the OpenShift installation guide (4.14).
They will be described in the next section.
4.3. Post installation steps
4.3.1. (Optional) Installing OpenShift Data Foundation
Red Hat OpenShift Data Foundation (ODF) has been validated as the persistent storage provider for SAP Data Intelligence. Please refer to the ODF documentation (4.12).
Please make sure to read and follow Disconnected Environment (4.12) if you install on a disconnected cluster.
4.3.2. (Optional) Installing NetApp Trident
NetApp Trident and StorageGRID have been validated for SAP Data Intelligence and OpenShift. More details can be found at SAP Data Intelligence on OpenShift 4 with NetApp Trident.
4.3.3. Configuring SDI Compute nodes
Some SDI components require changes to the OS level of Compute nodes. These could impact other workloads running on the same cluster. To prevent that from happening, it is recommended to dedicate a set of nodes to SDI workload. The following needs to be done:
- Chosen nodes must be labelled e.g. using the
node-role.kubernetes.io/sdi=""
label. - MachineConfigs specific to SDI need to be created; they will be applied only to the selected nodes. SDI Observer Operator can be configured to perform this operation as introduced in the next section with
manageSDINodeConfig
parameter set to "true". - MachineConfigPool must be created to associate the chosen nodes with the newly created MachineConfigs. No change will be done to the nodes until this point.SDI Observer Operator can be configured to perform this operation as introduced in the next section with the
manageSDINodeConfig
parameter set to "true". - (optional) Apply a node selector to
sdi
,sap-slcbridge
anddatahub-system
projects. SDI Observer Operator can be configured to perform this operation, as introduced in the next section, with theSDINodeLabel
parameter.
4.3.3.1. Labelling the Compute nodes for SAP Data Intelligence
Choose Compute nodes for the SDI workload and label them from the Management host like this:
# oc label node/sdi-worker{1,2,3} node-role.kubernetes.io/sdi=""
This step, combined with the SDI Observer Operator functionality, ensures that SAP DI-related workloads in specific namespaces are run on the labeled nodes. The SDI Observer adds a node selector annotation to the relevant SAP DI namespaces (e.g., openshift.io/node-selector: node-role.kubernetes.io/sdi=), which typically include the SAP DI namespace, datahub-system, and sap-slcbridge namespaces.
4.4. Installing SDI Observer Operator
SDI Observer Operator monitors SDI and SLC Bridge namespaces and applies changes to SDI deployments to allow SDI to run on OpenShift. Among other things, it does the following:
- Adds additional persistent volume to
vsystem-vrep
StatefulSet to allow it to run on the RHCOS system. - Exposes SDI System Management service.
- Exposes SLC Bridge service.
- Configure NFS exports for SDI
vsystem-vrep
. - Configure host path mounts for SDI diagnostic pods.
- Configure node selector in namespace.
- Add roles and rolebindings for service accounts in the SDI namespace.
It is deployed as an OpenShift Community Operator. Its behaviour is controlled by the SDIObserver instance configuration, which can be created after the deployment of the Observer Operator.
Please refer to its documentation for the complete list of issues that it currently attempts to solve.
You can install SAP Data Intelligence Observer Operator using the Red Hat OpenShift Container Platform Operator Hub.
Prerequisites
- Access to an OpenShift Container Platform cluster using an account with cluster-admin and operator installation permissions.
4.4.1 Install SDI Observer Operator
You can install an SDI Observer Operator and an SDIObserver instance either from the command line or through the OpenShift web console. Both methods are detailed in the following section.
4.4.1.1 Install SDI Observer Operator Using command line interface
Procedure
- Subscribe to the sdi-observer-operator to install the operator
cat <<EOF | oc apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
labels:
operators.coreos.com/sap-data-intelligence-observer-operator.openshift-operators: ""
name: sap-data-intelligence-observer-operator
namespace: openshift-operators
spec:
channel: stable
installPlanApproval: Automatic
name: sap-data-intelligence-observer-operator
source: community-operators
sourceNamespace: openshift-marketplace
EOF
- Create the sdi-observer namespace
cat <<EOF | oc apply -f -
apiVersion: v1
kind: Namespace
metadata:
name: sdi-observer
spec: {}
EOF
- Create the sdiObserver instance
cat <<EOF | oc apply -f -
apiVersion: sdi.sap-redhat.io/v1alpha1
kind: SDIObserver
metadata:
name: sdiobserver-instance
namespace: sdi-observer
spec:
SDINodeLabel: node-role.kubernetes.io/sdi= # <-- Modify the node label if it's different than this one
manageSDINodeConfig: true
sdiNamespace: sdi # <-- Modify the namespace if SDI is not installed in "sdi" namespace
sdiVSystemRoute:
managementState: Managed
slcbNamespace: sap-slcbridge # <-- Modify the namespace if sap slc bridge is not installed in "sap-slcbridge" namespace
slcbRoute:
managementState: Managed
EOF
4.4.1.2 Install SDI Observer Operator using OpenShift web console
Procedure
- Log in to the
OpenShift Web Console
. - Click Operators → OperatorHub.
- Scroll or type
SAP Data Intelligence 3 - Observer Operator
into the Filter by keyword box to find the SAP Data Intelligence Observer Operator. - Click Install.
- Set the following options on the Install Operator page:
- Update Channel to be stable.
- Installation Mode as the default All namespaces on the cluster.
- Installed Namespace as the default Operator recommended namespace openshift-operators.
- Select Approval Strategy as Automatic or Manual.
- If you select Automatic updates, then the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention.
- If you select Manual updates, then the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator to a newer version.
- Ensure that the Enable option is selected for the Console plugin.
- Click Install.
After the installation of SDI Observer Operator is done. We can create SDIObserver instance using OpenShift Web Console:
Procedure
- In the OpenShift Web Console, click Operators → Installed Operators to view all the installed operators.
Ensure that the Project selected is your intended sdi observer installation namespace, e.g., "sdi-observer" (in this case, please make sure the namespace is created beforehand by using "oc new-project sdi-observer"). - Click on the SAP Data Intelligence 3 - Observer Operator, and then click Create instance of SDIObserver.
In the Create SDIObserver page, perform the following:
- Select Form view.
- Set the Name for the SDIObserver instance or use the default value.
- Select or unselect the checkbox of manageSDINodeConfig.
- Set the sdiNamespace to specify on which namespace SAP Data Intelligence will be installed.
- Click sdiVsystemRoute to change the managementState of the SAP Data Intelligence vSystem route.
- Set the slcbNamespace to specify in which namespace the SLC Bridge will be installed.
- Click slcbRoute to change the managementState of the SLC Bridge route.
- Set the SDINodeLabel for the SDI node label or use the default value
node-role.kubernetes.io/sdi=
. - Switch Configure via from Form view to YAML view to verify if the fields are set correctly, especially the namespace under the metadata section.
- Click Create.
Verification steps
To verify the final Status of the installed SDIObserver instance:
- In the OpenShift Web Console, navigate to Installed Operators → SAP Data Intelligence 3 - Observer Operator → SDIObserver.
- Click SDIObservers.
- Verify that the SDIObserver instance is created correctly.
4.4.2. Managing SDIObserver instance
You can change and manage the SDIObserver instance from the OpenShift Web Console.
- In the OpenShift Web Console, click Operators → Installed Operators to view all the installed operators.
Ensure that the Project selected is your intended sdi observer installation namespace, e.g., "sdi-observer". - Click on the SAP Data Intelligence 3 - Observer Operator, and then click the SDIObserver tab to find the existing SDIObserver Instance.
- Click on the existing SDIObserver instance, and then click the YAML tab, and you can modify the configuration accordingly.
Thespec.sdiVsystemroute.managementState
andspec.slcbRoute.managementState
can be set toManaged
,Unmanaged
orRemoved
according to operational requirements. - After the modification, click Save.
4.4.2.1. Monitor SDIObserver Logs and Handle Errors
- To check the SDIObserver pod logs, use the following command:
# oc logs -n openshift-operators $(kubectl get pods -n openshift-operators --no-headers -o custom-columns=":metadata.name" | grep '^observer-operator-controller-manager')
- If the observer-operator pod encounters errors or is stuck in an error status, delete the pod to restart the reconciliation process:
# oc delete pod -n openshift-operators $(kubectl get pods -n openshift-operators --no-headers -o custom-columns=":metadata.name" | grep '^observer-operator-controller-manager')
- Wait a few minutes, then recheck the logs to confirm that the reconciliation process has completed successfully.
4.4.3. (Optional) Enabling SDI on the control plane
If the control plane (or master nodes) shall be used for running SDI workload, in addition to the previous step, one needs to perform the following:
- Please make sure the control plane is schedulable
- Duplicate the machine configs for master nodes:
# oc get -o json mc -l machineconfiguration.openshift.io/role=sdi | jq '.items[] |
select((.metadata.annotations//{}) |
has("machineconfiguration.openshift.io/generated-by-controller-version") | not) |
.metadata |= ( .name |= sub("^(?<i>(\\d+-)*)(worker-)?"; "\(.i)master-") |
.labels |= {"machineconfiguration.openshift.io/role": "master"} )' | oc apply -f -
Note that you may see a couple of warnings ∇ if this has been done earlier.
- Make the master machine config pool inherit the PID limit changes:
# oc label mcp/master workload=sapdataintelligence
The following command can be used to wait until the change gets applied to all the worker nodes:
# oc wait mcp/master --all --for=condition=updated
4.4.4. Verifying node configuration
The following steps assume that the node-role.kubernetes.io/sdi=""
label has been applied to nodes running the SDI workload. All the commands will be executed on the Management host. All the diagnostic commands will be run in parallel on such nodes.
- Verify that the PID limit has been increased to 16384:
# oc get nodes -l node-role.kubernetes.io/sdi= -o name | \
xargs -P 6 -n 1 -i oc debug $ocDebugArgs {} -- chroot /host /bin/bash -c \
"cat /etc/kubernetes/kubelet.conf" | jq '.podPidsLimit'
- Verify that the kernel modules have been loaded:
# oc get nodes -l node-role.kubernetes.io/sdi= -o name | \
xargs -P 6 -n 1 -i oc debug {} -- chroot /host /bin/sh -c \
"lsmod | awk 'BEGIN {ORS=\":\t\"; print ENVIRON[\"HOSTNAME\"]; ORS=\",\"}
/^(nfs|ip_tables|iptable_nat|[^[:space:]]+(REDIRECT|owner|filter))/ {
print \$1
}'; echo" 2>/dev/null
An example output could look like this:
sdi-worker2: iptable_filter,iptable_nat,xt_owner,xt_REDIRECT,nfsv4,nfs,nfsd,nfs_acl,ip_tables,
sdi-worker3: iptable_filter,iptable_nat,xt_owner,xt_REDIRECT,nfsv4,nfs,nfsd,nfs_acl,ip_tables,
sdi-worker1: iptable_filter,iptable_nat,xt_owner,xt_REDIRECT,nfsv4,nfs,nfsd,nfs_acl,ip_tables,
If any of the following modules is missing on any of the SDI nodes, the module loading does not work: iptable_nat
, nfsv4
, nfsd
, ip_tables
, xt_owner
To further debug missing modules, one can also execute the following command:
# oc get nodes -l node-role.kubernetes.io/sdi= -o name | \
xargs -P 6 -n 1 -i oc debug {} -- chroot /host /bin/bash -c \
"( for service in {sdi-modules-load,systemd-modules-load}.service; do \
printf '%s:\t%s\n' \$service \$(systemctl is-active \$service); \
done; find /etc/modules-load.d -type f \
-regex '.*\(sap\|sdi\)[^/]+\.conf\$' -printf '%p\n';) | \
awk '{print ENVIRON[\"HOSTNAME\"]\":\t\"\$0}'" 2>/dev/null
Please make sure that both systemd services are active
and at least one *.conf
file is listed for each host, as shown in the following example output:
sdi-worker3: sdi-modules-load.service: active
sdi-worker3: systemd-modules-load.service: active
sdi-worker3: /etc/modules-load.d/sdi-dependencies.conf
sdi-worker1: sdi-modules-load.service: active
sdi-worker1: systemd-modules-load.service: active
sdi-worker1: /etc/modules-load.d/sdi-dependencies.conf
sdi-worker2: sdi-modules-load.service: active
sdi-worker2: systemd-modules-load.service: active
sdi-worker2: /etc/modules-load.d/sdi-dependencies.conf
4.4.5. Deploying persistent storage provider
Unless your platform already offers a supported persistent storage provider, one needs to be deployed. Please refer to Understanding persistent storage (4.14) for an overview of possible options.
On OpenShift, one can deploy OpenShift Data Foundation (ODF) (4.14) running converged on OpenShift nodes, providing both persistent volumes and object storage. Please refer to ODF Planning your Deployment (4.12) and Deploying OpenShift Data Foundation (4.12) for more information and installation instructions.
For the NFS based persistent storage, the dynamic storage provisioning method is a prerequisite. Please refer to the following article for detailed information, "How do I create a storage class for NFS dynamic storage provisioning in an OpenShift environment?".
4.4.6. Configuring S3 access and bucket
Object storage is required for the following features of SDI:
- backup and restore (previously checkpoint store) feature provides regular backups of SDI service data
- SDL Data Lake connection (3.3) for the machine learning scenarios
Several interfaces to object storage are supported by SDI. S3 interface is one of them. Please take a look at Checkpoint Store Type at Required Input Parameters (3.3) for the complete list. SAP help page covers the preparation of object stores (3.3).
Backup and restore can be enabled against ODF NooBaa's S3 endpoint as long as ODF is of version 4.6.4 or newer, or against the RADOS Object Gateway S3 endpoint when ODF is deployed in external mode.
4.4.6.1. Using NooBaa or RADOS object gateway S3 endpoint as object storage
ODF contains a NooBaa object data service for hybrid and multi cloud environments, which provides an S3 API one can use with SAP Data Intelligence. Starting from ODF release 4.6.4, it can also be used for SDI's backup and restore functionality. Alternatively, the functionality can be enabled against RADOS Object Gateway S3 endpoint (from now on, just RGW) which is available when ODF is deployed in the external mode (4.12).
For SDI, one needs to provide the following:
- S3 host URL prefixed either with
https://
orhttp://
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
- bucket name
NOTE In the case of https://
, the endpoint must be secured by certificates signed by a trusted certificate authority. Self-signed CAs will not work out of the box as of now.
Once ODF is deployed, one can create access keys and buckets using one of the following:
- (internal mode only) via NooBaa Management Console, by default exposed at
noobaa-mgmt-openshift-storage.apps.<cluster_name>.<base_domain>
- (both internal and external modes) via CLI with
mksdibuckets
script
In both cases, the S3 endpoint provided to SAP Data Intelligence cannot be secured with a self-signed certificate as of now. Unless the endpoints are secured with a properly signed certificate, one must use an insecure HTTP connection. Both NooBaa and RGW come with an insecure service reachable from inside the cluster (within the SDN), which cannot be resolved from outside the cluster unless exposed via, e.g., a route.
The following two URLs are example endpoints on an OpenShift cluster with ODF deployed.
http://s3.openshift-storage.svc.cluster.local
- NooBaa S3 Endpoint is always availablehttp://rook-ceph-rgw-ocs-external-storagecluster-cephobjectstore.openshift-storage.svc.cluster.local:8080
- RGW endpoint that shall be preferably used when ODF is deployed in the external mode
4.4.6.1.1. Creating an S3 bucket using CLI
The bucket can be created with the command below, executed from the Management host. Be sure to switch to the appropriate project or namespace (e.g., sdi
) first before executing the following command or appending parameters -n SDI_NAMESPACE
to it.
- (Create buckets)
# bash <(curl -s https://raw.githubusercontent.com/redhat-sap/sap-data-intelligence/master/utils/mksdibuckets)
- (List buckets)
# bash <(curl -s https://raw.githubusercontent.com/redhat-sap/sap-data-intelligence/master/utils/mksdibuckets) list
Example output:
Bucket claim namespace/name: sdi/sdi-checkpoint-store (Status: Bound, Age: 7m33s)
Cluster internal URL: http://s3.openshift-storage.svc.cluster.local
Bucket name: sdi-checkpoint-store-ef4999e0-2d89-4900-9352-b1e1e7b361d9
AWS_ACCESS_KEY_ID: LQ7YciYTw8UlDLPi83MO
AWS_SECRET_ACCESS_KEY: 8QY8j1U4Ts3RO4rERXCHGWGIhjzr0SxtlXc2xbtE
Bucket claim namespace/name: sdi/sdi-data-lake (Status: Bound, Age: 7m33s)
Cluster internal URL: http://s3.openshift-storage.svc.cluster.local
Bucket name: sdi-data-lake-f86a7e6e-27fb-4656-98cf-298a572f74f3
AWS_ACCESS_KEY_ID: cOxfi4hQhGFW54WFqP3R
AWS_SECRET_ACCESS_KEY: rIlvpcZXnonJvjn6aAhBOT/Yr+F7wdJNeLDBh231
For more information and options, run the command with --help
.
The example above uses ODF NooBaa's S3 endpoint, which is always the preferred choice for ODF internal mode.
The values of the claim sdi-checkpoint-store
shall be passed to the following SLC Bridge parameters during SDI's installation in order to enable backup and restore (previously known as checkpoint store) functionality.
Parameter | Example value |
---|---|
Object Store Type | S3 compatible object store |
Access Key | LQ7YciYTw8UlDLPi83MO |
Secret Key | 8QY8j1U4Ts3RO4rERXCHGWGIhjzr0SxtlXc2xbtE |
Endpoint | http://s3.openshift-storage.svc.cluster.local |
Path | sdi-checkpoint-store-ef4999e0-2d89-4900-9352-b1e1e7b361d9 |
Disable Certificate Validation | Yes |
4.4.6.1.2. Increasing object bucket limits
NOTE This is needed only for RGW (ODF external mode)
When performing checkpoint store validation during SDI installation, the installer will create a temporary bucket. For that to work with the RGW, the bucket's owner limit on maximum allocatable buckets needs to be increased. The limit is set to 1 by default.
You can use the following command to perform the needed changes for the bucket assigned to the backup and restore (checkpoint store). Please execute it on the management node of the external Red Hat Ceph Storage cluster (or on the host where the external RGW service runs). The last argument is the "Bucket name", not the "Bucket claim name".
# bash <(curl -s https://raw.githubusercontent.com/redhat-sap/sap-data-intelligence/master/utils/rgwtunebuckets) \
sdi-checkpoint-store-ef4999e0-2d89-4900-9352-b1e1e7b361d9
For more information and additional options, append the --help
parameter at the end.
4.4.7. Setting up a container image registry
If you haven't done so already, please follow the Container image registry prerequisite.
NOTE It is now required to use a registry secured by TLS for SDI. Plain HTTP
will not do.
If the registry is signed by a proper trusted (not self-signed) certificate, this may be skipped.
There are two ways to make OpenShift trust an additional registry using certificates signed by a self-signed certificate authority:
- (recommended) Update the CA certificate trust in OpenShift's image configuration
- (less secure) Mark the registry as insecure
4.4.8. Configuring the OpenShift cluster for SDI
4.4.8.1. Becoming a cluster-admin
Many commands below require cluster admin privileges. To become a cluster-admin, you can do one of the following:
- Use the
auth/kubeconfig
generated in the working directory during the installation of the OpenShift cluster:
INFO Install complete!
INFO Run 'export KUBECONFIG=<your working directory>/auth/kubeconfig' to manage the cluster with 'oc', the OpenShift CLI.
INFO The cluster is ready when 'oc login -u kubeadmin -p <provided>' succeeds (wait a few minutes).
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.demo1.openshift4-beta-abcorp.com
INFO Login to the console with user: kubeadmin, password: <provided>
# export KUBECONFIG=working_directory/auth/kubeconfig
# oc whoami
system:admin
- As a
system:admin
user or a member ofcluster-admin
group, make another user a cluster admin to allow him to perform the SDI installation:
- As a cluster-admin, configure the authentication (4.14) and add the desired user (e.g.
sdiadmin
). - As a cluster-admin, grant the user permission to administer the cluster:
# oc adm policy add-cluster-role-to-user cluster-admin sdiadmin
You can learn more about the cluster-admin role in the Cluster Roles and Local Roles article (4.14).
5. Installing SDI on OpenShift
5.1. Installing Software Lifecycle Container Bridge
Please follow the guide Installing SAP Data Intelligence using the SLC Bridge (3.3).
5.1.1. Important parameters
Parameter | Condition | Description |
---|---|---|
Mode | Always | Make sure to choose the Expert Mode. |
Address of the Container Image Repository | Always | The URL of the container image registry. |
Image registry username | if … ‡ | Please refer to your registry configuration. |
Image registry password | if … ‡ | Please refer to your registry configuration. |
Namespace of the SLC Bridge | Always | If you override the default (sap-slcbridge ), make sure to deploy SDIObserver instance with the corresponding SLCB namespace parameter. |
Service Type | SLC Bridge Base installation | On vSphere, make sure to use NodePort . On AWS, please use LoadBalancer . |
Cluster No Proxy | Required in conjunction with the HTTPS Proxy value | Make sure to do this according to the Configuring HTTP Proxy for the SLC Bridge section. |
‡ If the registry requires authentication, Red Hat Quay does it.
For more details, please refer to Configuring the cluster-wide proxy (4.14).
On a NAT'd on-premise cluster, in order to access the slcbridgebase-service
NodePort
service, one needs to have either direct access to one of the SDI Compute nodes or modify an external load balancer to add an additional route to the service.
5.1.2. Installing SLC Bridge
Please install SLC Bridge according to Making the SLC Bridge Base available on Kubernetes (3.3) while paying attention to the notes on the installation parameters.
5.1.2.1. Exposing SLC Bridge with OpenShift Ingress Controller
For SLC Bridge, the only possible type of TLS termination is passthrough
unless the Ingress Controller is configured to use globally trusted certificates.
It is recommended to let the SDI Observer Operator manage the route creation and updates. If the SDI Observer Instance has been deployed with spec.sdislcbRoute.managementState: Managed
, this section can be skipped.
After a while, the bridge will be available at https://sap-slcbridge-sap-slcbridge.apps.<cluster_name>.<base_domain>/docs/index.html
. You can wait for the route's availability like this:
# oc get route -w -n "${SLCB_NAMESPACE:-sap-slcbridge}"
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
sap-slcbridge <ROUTE_NAME>-<SLCB_NAMESPACE>.apps.<cluster_name>.<base_domain> slcbridgebase-service <all> passthrough/Redirect None
5.1.2.1.1. Exposing SLC Bridge manually with Ingress
Alternatively, you can expose the SLC Bridge manually with this approach.
- Look up the
slcbridgebase-service
service:
# oc project "${SLCB_NAMESPACE:-sap-slcbridge}" # switch to the Software Lifecycle Bridge project
# oc get services | grep 'NAME\|slcbridge'
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
slcbridgebase-service NodePort 172.30.206.105 <none> 32455:31477/TCP 14d
- Create the route for the service:
# oc create route passthrough sap-slcbridge --service=slcbridgebase-service \
--insecure-policy=Redirect --dry-run=client -o json | \
oc annotate --local -f - haproxy.router.openshift.io/timeout=10m -o json | oc apply -f -
You can also set your desired hostname with the --hostname
parameter. Make sure it resolves to the router's IP.
- Get the generated hostname:
# oc get route
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
vsystem vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain> vsystem vsystem passthrough/Redirect None
-
Make sure to configure your external load balancer to increase the timeout for WebSocket connections for this particular hostname by at least 10 minutes. For example, in HAProxy, it would be
timeout tunnel 10m
. -
Access the System Management service at
https://vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain>
to verify.
5.1.2.2. Using an external load balancer to access SLC Bridge's NodePort
NOTE This is applicable only when "Service Type" is set to "NodePort".
Once the SLC Bridge is deployed, its NodePort
shall be determined in order to point the load balancer at it.
# oc get svc -n "${SLCB_NAMESPACE:-sap-slcbridge}" slcbridgebase-service -o jsonpath='{.spec.ports[0].nodePort}{"\n"}'
31875
The load balancer shall point at all the Compute nodes running SDI workloads. The following is an example of a HAProxy load balancer:
# # in the example, the <cluster_name> is "boston" and <base_domain> is "ocp.vslen"
# cat /etc/haproxy/haproxy.cfg
....
frontend slcb
bind *:9000
mode tcp
option tcplog
# # commented blocks are useful for multiple OpenShift clusters or multiple SLC Bridge services
#tcp-request inspect-delay 5s
#tcp-request content accept if { req_ssl_hello_type 1 }
use_backend boston-slcb #if { req_ssl_sni -m end -i boston.ocp.vslen }
#use_backend raleigh-slcb #if { req_ssl_sni -m end -i raleigh.ocp.vslen }
backend boston-slcb
balance source
mode tcp
server sdi-worker1 sdi-worker1.boston.ocp.vslen:31875 check
server sdi-worker2 sdi-worker2.boston.ocp.vslen:31875 check
server sdi-worker3 sdi-worker3.boston.ocp.vslen:31875 check
backend raleigh-slcb
....
The SLC Bridge can then be accessed at the URL https://boston.ocp.vslen:9000/docs/index.html
as long as boston.ocp.vslen
resolves correctly to the load balancer's IP.
5.2. SDI installation parameters
Please follow SAP's guidelines on configuring the SDI while paying attention to the following additional comments:
Name | Condition | Recommendation |
---|---|---|
Kubernetes Namespace | Always | Must match the project name chosen in the Project Setup (e.g. sdi ). |
Installation Type | Installation or Update | Choose Advanced Installation if you need to specify that you want to choose a particular storage class or there is no default storage class (4.14) / (4.10) set or you want to deploy multiple SDI instances on the same cluster. |
Container Image Repository | Installation | Must be set to the container image registry. |
Cluster Proxy Settings | Advanced Installation or Update | Choose yes if a local HTTP(S) proxy must be used to access external web resources. |
Cluster No Proxy | When Cluster Proxy Settings is configured. |
Please refer to the HTTP proxy configuration. |
Backup Configuration | Installation or Upgrade from a system in which backups are not enabled | For a production environment, please choose yes. ⁴ |
Checkpoint Store Configuration | Installation | Recommended for production deployments. If backup is enabled, it is enabled by default. |
Checkpoint Store Type | If Checkpoint Store Configuration parameter is enabled. | Set to S3 compatible object store if using, for example, ODF or NetApp StorageGRID as the object storage. See Using NooBaa as object storage gateway or NetApp StorageGRID for more details. |
Disable Certificate Validation | If Checkpoint Store Configuration parameter is enabled. | Please choose yes if using the HTTPS for your object storage endpoint secured with a certificate having a self-signed CA. For ODF NooBaa, you can set it to no. |
Checkpoint Store Validation | Installation | Please make sure to validate the connection during the installation time. Otherwise, if an incorrect value is supplied, the installation will fail at a later point. |
Container Registry Settings for Pipeline Modeler | Advanced Installation | Shall be changed if the same registry is used for more than one SAP Data Intelligence instance. Either another <registry> or a different <prefix> or both will do. |
StorageClass Configuration | Advanced Installation | Configure this if you want to choose different dynamic storage provisioners for different SDI components or if there's no default storage class (4.14) / (4.10) set or you want to choose a non-default storage class for the SDI components. |
Default StorageClass | Advanced Installation and if storage classes are configured | Set this if there's no default storage class (4.14) / (4.10) set or you want to choose non-default storage class for the SDI components. |
Enable Kaniko Usage | Advanced Installation | Must be enabled on OpenShift 4. |
Container Image Repository Settings for SAP Data Intelligence Modeler | Advanced Installation or Upgrade | If using the same registry for multiple SDI instances, choose "yes". |
Container Registry for Pipeline Modeler | Advanced Installation and if the "Use different one" option is selected in the previous selection. | If using the same registry for multiple SDI instances, it is required to use either a different prefix (e.g., local.image.registry:5000/mymodelerprefix2 ) or a different registry. |
Loading NFS Modules | Advanced Installation | Feel free to say "no". This is no longer a concern as long as the loading of the needed kernel modules has been configured. |
Additional Installer Parameters | Advanced Installation | Please include -e vsystem.vRep.exportsMask=true . Or ff is omitted and SDIObserver is configured accordingly, it will apply this parameter on your behalf. |
⁴ Note that the validated S3 API endpoint providers are ODF's NooBaa 4.6.4 or newer, ODF 4.6 in external mode and NetApp StorageGRID.
5.3. Installing SDI
Please follow the official procedure according to Install SAP Data Intelligence with SLC Bridge in a Cluster with Internet Access (3.3).
5.4. Post installation steps
5.4.1. (Optional) Exposing SDI services externally
There are multiple possibilities for making SDI services accessible outside the cluster. Compared to Kubernetes, OpenShift offers additional methods, which is recommended for most of the scenarios, including the SDI System Management service. It's based on the OpenShift Ingress Operator (4.14).
For SAP Vora Transaction Coordinator and SAP HANA Wire, please use the official suggested method available for your environment (3.3).
5.4.1.1. Using OpenShift Ingress Operator
NOTE Instead of using this manual approach, it is now recommended to let the SDI Observer Operator manage the route creation and updates instead. If the SDI Observer Instance has been created with spec.sdiVSystemRoute.managementState: Managed
, this section can be skipped.
Or please continue with the manual route creation.
OpenShift allows you to access the Data Intelligence services via Ingress Controllers (4.14) as opposed to the regular NodePorts (4.14). For example, instead of accessing the vsystem service via https://worker-node.example.com:32322
, after the service exposure, you will be able to access it at https://vsystem-sdi.apps.<cluster_name>.<base_domain>
. This is an alternative to the official guide documentation to Expose the Service On Premise (3.3).
There are two kinds of routes secured with TLS. The reencrypt
kind, allows for a custom signed or self-signed certificate to be used. The other kind is passthrough
, which uses the pre-installed certificate generated by or passed to the installer.
5.4.1.1.1. Exporting services with a reencrypt route
With this kind of route, different certificates are used on the client and service sides of the route. The router stands in the middle and re-encrypts the communication coming from either side using a certificate corresponding to the other side. In this case, the client side is secured by a provided certificate and the service side is encrypted with the original certificate generated or passed to the SAP Data Intelligence installer. This is the same kind of route SDI Observer creates automatically.
The reencrypt route allows for securing the client connection with a properly signed certificate.
- Look up the
vsystem
service:
# oc project "${SDI_NAMESPACE:-sdi}" # switch to the Data Intelligence project
# oc get services | grep "vsystem "
vsystem ClusterIP 172.30.227.186 <none> 8797/TCP 19h
When exported, the resulting hostname will look like vsystem-${SDI_NAMESPACE}.apps.<cluster_name>.<base_domain>
. However, an arbitrary hostname can be chosen instead, as long as it resolves correctly to the IP of the router.
- Get, generate or use the default certificates for the route. In this example, the default self-signed certificate used by router is used to secure the connection between the client and OpenShift's router. The CA certificate for clients can be obtained from the
router-ca
secret located in theopenshift-ingress-operator
namespace:
# oc get secret -n openshift-ingress-operator -o json router-ca | \
jq -r '.data as $d | $d | keys[] | select(test("\\.crt$")) | $d[.] | @base64d' >router-ca.crt
- Obtain the SDI's root certificate authority bundle generated at the SDI's installation time. The generated bundle is available in the
ca-bundle.pem
secret in thesdi
namespace.
# oc get -n "${SDI_NAMESPACE:-sdi}" -o go-template='{{index .data "ca-bundle.pem"}}' \
secret/ca-bundle.pem | base64 -d >sdi-service-ca-bundle.pem
- Create the reencrypt route for the vsystem service like this:
# oc create route reencrypt -n "${SDI_NAMESPACE:-sdi}" --dry-run -o json \
--dest-ca-cert=sdi-service-ca-bundle.pem --service vsystem \
--insecure-policy=Redirect | \
oc annotate --local -o json -f - haproxy.router.openshift.io/timeout=2m | \
oc apply -f -
# oc get route
NAME HOST/PORT SERVICES PORT TERMINATION WILDCARD
vsystem vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain> vsystem vsystem reencrypt/Redirect None
- Verify the connection:
# # use the HOST/PORT value obtained from the previous command instead
# curl --cacert router-ca.crt https://vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain>/
5.4.1.1.2. Exporting services with a passthrough route
With the passthrough
route, the communication is encrypted by the SDI service's certificate all the way to the client.
NOTE If possible, please prefer the reencrypt
route because the hostname of vsystem certificate cannot be verified by clients, as can be seen in the following output:
# oc get -n "${SDI_NAMESPACE:-sdi}" -o go-template='{{index .data "ca-bundle.pem"}}' \
secret/ca-bundle.pem | base64 -d >sdi-service-ca-bundle.pem
# openssl x509 -noout -subject -in sdi-service-ca-bundle.pem
subject=C = DE, ST = BW, L = Walldorf, O = SAP, OU = Data Hub, CN = SAPDataHub
- Look up the
vsystem
service:
# oc project "${SDI_NAMESPACE:-sdi}" # switch to the Data Intelligence project
# oc get services | grep "vsystem "
vsystem ClusterIP 172.30.227.186 <none> 8797/TCP 19h
- Create the route:
# oc create route passthrough --service=vsystem --insecure-policy=Redirect
# oc get route
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
vsystem vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain> vsystem vsystem passthrough/Redirect None
You can modify the hostname with --hostname
parameter. Make sure it resolves to the router's IP.
- Access the System Management service at
https://vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain>
to verify.
5.4.1.2. Using NodePorts
NOTE For OpenShift, an exposure using routes is preferred although only possible for the System Management service (aka vsystem
).
Exposing SAP Data Intelligence vsystem
- Either with an auto-generated NodePort:
# oc expose service vsystem --type NodePort --name=vsystem-nodeport --generator=service/v2
# oc get -o jsonpath='{.spec.ports[0].nodePort}{"\n"}' services vsystem-nodeport
30617
- Or with a specific NodePort (e.g., 32123):
# oc expose service vsystem --type NodePort --name=vsystem-nodeport --generator=service/v2 --dry-run -o yaml | \
oc patch -p '{"spec":{"ports":[{"port":8797, "nodePort": 32123}]}}' --local -f - -o yaml | oc apply -f -
The original service remains accessible on the same ClusterIP:Port
as before. Additionally, it is now accessible from outside of the cluster under the NodePort.
[For SDI 3.2 Only] Exposing SAP Vora Transaction Coordinator and HANA Wire
# oc expose service vora-tx-coordinator-ext --type NodePort --name=vora-tx-coordinator-nodeport --generator=service/v2
# oc get -o jsonpath='tx-coordinator:{"\t"}{.spec.ports[0].nodePort}{"\n"}hana-wire:{"\t"}{.spec.ports[1].nodePort}{"\n"}' \
services vora-tx-coordinator-nodeport
tx-coordinator: 32445
hana-wire: 32192
The output shows the generated NodePorts for the newly exposed services.
5.4.2. Configuring the connection to Data Lake
Please follow the official post-installation instructions at Configure the Connection to DI_DATA_LAKE
(3.3).
In case the ODF is used as a backing object storage provider, please make sure to use the HTTP service endpoint as documented in Using NooBaa or RADOS Object Gateway S3 endpoint as object storage.
Based on the example output in that section, the configuration may look like this:
Parameter | Value |
---|---|
Connection Type | SDL |
Id | DI_DATA_LAKE |
Object Storage Type | S3 |
Endpoint | http://s3.openshift-storage.svc.cluster.local |
Access Key ID | cOxfi4hQhGFW54WFqP3R |
Secret Access Key | rIlvpcZXnonJvjn6aAhBOT/Yr+F7wdJNeLDBh231 |
Root Path | sdi-data-lake-f86a7e6e-27fb-4656-98cf-298a572f74f3 |
5.4.3. Validating SDI
Validate the SDI installation on OpenShift to make sure everything works as expected. Please follow the instructions in Testing Your Installation (3.3).
5.4.3.1. Logging on to SAP Data Intelligence launchpad
In case the vsystem
service has been exposed using a route, the URL can be determined like this:
# oc get route -n "${SDI_NAMESPACE:-sdi}"
NAME HOST/PORT SERVICES PORT TERMINATION WILDCARD
vsystem vsystem-<SDI_NAMESPACE>.apps.<cluster_name>.<base_domain> vsystem vsystem reencrypt None
The HOST/PORT
value needs to be prefixed with https://
, for example:
https://vsystem-sdi.apps.boston.ocp.vslen
5.4.3.2. Checking your machine learning setup
In order to upload training and test datasets using ML Data Manager, the user needs to be assigned the app.datahub-app-data.fullAcces
(as of 3.2) or sap.dh.metadata
(up to 3.1) policy. Please make sure to follow Using SAP Data Intelligence Policy Management (3.3) to assign the policies to the users that need them.
5.4.4. Configuring additional tenants
When a new tenant is created (using, e.g., Manage Clusters instructions (3.3)) it is not configured to work with the container image registry. Therefore, the Pipeline Modeler is unusable and will fail to start until configured.
There are a few steps that need to be performed for each new tenant:
- import CA certificate for the registry via SDI Connection Manager if the CA certificate is self-signed
- as long as a different registry for modeler is used, the pull secret needs to be imported into the SDI_NAMESPACE
- create and import credential secret using SDI System Management and update the modeler secret if the container image registry requires authentication
If the Red Hat Quay is used, please follow the Configuring additional SDI tenants. Please make sure to execute the official instructions in the following articles according to your registry configuration:
- Provide Access Credentials for a Password Protected Container Registry (3.3) (as long as your registry for the Pipeline Modeler uses TLS with a self-signed CA).
- (3.3) (as long as your registry requires authentication).
6. Upgrading OpenShift Container Platform
This section is useful as a guide for performing OpenShift upgrades to the latest asynchronous releaseⁿ of the same minor version or to the newer minor release supported by the running SDI instance without upgrading SDI itself.
6.1. Pre-upgrade procedures
- Make yourself familiar with (4.13 ⇒ 4.14).
- Plan for SDI downtime.
- Make sure to re-configure SDI Compute nodes.
6.1.1. Stopping SAP Data Intelligence
In order to speed up the cluster upgrade and/or to ensure SDI's consistency, it is possible to stop SDI before performing the upgrade.
The procedure is outlined in the official Administration Guide (3.3). In short, the command is:
# oc -n "${SDI_NAMESPACE}" patch datahub default --type='json' -p '[
{"op":"replace","path":"/spec/runLevel","value":"Stopped"}]'
6.2. Upgrading OpenShift
The following instructions outline the process of an OpenShift upgrade to a minor release 2 versions higher than the current one. If only an upgrade to the latest asynchronous releaseⁿ of the same minor version is desired, please skip steps 5 and 6.
- Upgrade OpenShift to a higher minor release or the latest asynchronous release (⇒ 4.14).
- If you have OpenShift Data Foundation deployed, update ODF to the latest supported release for the current OpenShift release according to the interoperability guide.
- Update OpenShift client tools on the Management host to match the target ※ OpenShift release. On RHEL 8, one can do it like this:
# current=4.10; new=4.14
# sudo subscription-manager repos \
--disable=rhocp-${current}-for-rhel-8-x86_64-rpms --enable=rhocp-${new}-for-rhel-8-x86_64-rpms
# sudo dnf update -y openshift-clients
- Upgrade OpenShift to a higher minor release or the latest asynchronous release (⇒ 4.14) ⁿ.
- If you have OpenShift Data Foundation deployed, update ODF to the latest supported release for the current OpenShift release according to the interoperability guide.
※ For the initial OpenShift release 4.X
, the target release is 4.(X+2)
; if performing just the latest asynchronous releaseⁿ upgrade, the target release is 4.X
6.3. Post-upgrade procedures
- Start SAP Data Intelligence as outlined in the official Administration Guide (3.3). In short, the command is:
# oc -n "${SDI_NAMESPACE}" patch datahub default --type='json' -p '[
{"op":"replace","path":"/spec/runLevel","value":"Started"}]'
7. Upgrading or updating SAP Data Intelligence
NOTE This section covers an upgrade of SAP Data Intelligence to a newer minor, micro or patch release. Sections related only to the former or the latter will be annotated with the following annotations:
- (upgrade) to denote a section specific to an upgrade from Data Intelligence to a newer minor release (
3.X ⇒ 3.(X+1)
) - (update) to denote a section specific to an update of Data Intelligence to a newer micro/patch release (
3.X.Y ⇒ 3.X.(Y+1)
) - annotation-free are sections relating to both
The following steps must be performed in the given order: Unless an OpenShift upgrade is needed, the steps marked with (ocp-upgrade) can be skipped.
7.1. Pre-upgrade or pre-update procedures
- Make sure to get familiar with (3.2 ⇒ 3.3).
- (ocp-upgrade) Make yourself familiar with (4.13 ⇒ 4.14).
- Plan for some downtime.
- Make sure to re-configure SDI Compute nodes.
7.1.1. Executing SDI's pre-upgrade procedures
Please follow (3.2 ⇒ 3.3).
7.1.1.1. Automated route removal
SDI Observer Operator now allows for the creation and updates of vsystem routes for external access. It takes care of updating the route's destination certificate during SDI's update. It can also be instructed to keep the route deleted, which is useful during SDI updates. You can instruct the SDI Observer Operator to delete the route like this:
- In the OpenShift Web Console, click Operators → Installed Operators to view all the installed operators. Ensure that the Project selected is your intended sdi observer installation namespace, e.g. "sdi-observer".
- Click on the SAP Data Intelligence 3 - Observer Operator, and then click SDIObserver tab to find the existing SDIObserver Instance.
- Click on the existing SDIObserver instance, and then click YAML tab, and you can modify the configuration accordingly.
- Set
spec.sdiVsystemroute.managementState
toRemoved
. - After the modification, click Save.
7.1.1.2. Manual route removal
If you exposed the vsystem service using routes, delete the route:
# # note the hostname in the output of the following command
# oc get route -n "${SDI_NAMESPACE:-sdi}"
# # delete the route
# oc delete route -n "${SDI_NAMESPACE:-sdi}" --all
7.2. Updating or upgrading SDI
7.2.1. Updating Software Lifecycle Container Bridge
Before updating the SLC Bridge, please consider exposing it via Ingress Controller.
If you decide to continue using the NodePort service load-balanced by an external load balancer, make sure to note down the current service, NodePort:
# oc get -o jsonpath='{.spec.ports[0].nodePort}{"\n"}' -n sap-slcbridge \
svc/slcbridgebase-service
31555
Please follow the official documentation (3.3) to obtain the binary and update its resources on OpenShift cluster.
If exposed via the Ingress Controller, you can skip the next step. Otherwise, re-set the nodePort to the previous value so no changes on the load-balancer side are necessary.
# nodePort=31555 # change your value to the desired one
# oc patch --type=json -n sap-slcbridge svc/slcbridgebase-service -p '[{
"op":"add", "path":"/spec/ports/0/nodePort","value":'"$nodePort"'}]'
7.2.2. Upgrading SAP Data Intelligence to a newer minor release
Execute the SDI upgrade according to the official instructions (DH 3.2 ⇒ 3.3).
7.3. Upgrading OpenShift
Depending on the target SDI release, OpenShift clusters must be upgraded either to a newer minor release or to the latest asynchronous releaseⁿ for the current minor release.
Upgraded/Current SDI Releases | Desired and Validated OpenShift Releases |
---|---|
3.3 | 4.14 |
3.3 | 4.12 |
If the current OpenShift release is two or more releases behind the desired one, the OpenShift cluster must be upgraded iteratively to each successive minor release until the desired one is reached.
- (Optional) Stop the SAP Data Intelligence, as it will speed up the cluster update and ensure SDI's consistency.
-
Make sure to follow the official upgrade instructions for your upgrade path:
-
When on OpenShift 4.13, please make sure to set
spec.sdiVsystemroute.managementState
toRemoved
until the SDI's update is finished. -
(Optional) Start the SAP Data Intelligence again if the desired OpenShift release has been reached.
- Upgrade OpenShift client tools on the Management host. The example below can be used for RHEL 8:
# current=4.10; new=4.14
# sudo subscription-manager repos \
--disable=rhocp-${current}-for-rhel-8-x86_64-rpms --enable=rhocp-${new}-for-rhel-8-x86_64-rpms
# sudo dnf update -y openshift-clients
7.4. SAP Data Intelligence post-upgrade procedures
-
Execute the Post-Upgrade Procedures for the SDH (3.3).
-
Re-create the route for the vsystem service using one of the following methods:
- (Recommended) Instruct the SDI Observer Operator to manage the route:
# oc set env -n "${NAMESPACE:-sdi-observer}" dc/sdi-observer MANAGE_VSYSTEM_ROUTE=true
# # wait for the observer to get re-deployed
# oc rollout status -n "${NAMESPACE:-sdi-observer}" -w dc/sdi-observer
- Follow Expose SDI services externally to recreate the route manually from scratch.
7.5. Validating SAP Data Intelligence
Validate the SDI installation on OpenShift to make sure everything works as expected. Please follow the instructions in Testing Your Installation (3.3).
8. Appendix
8.1. Uninstalling SDI
Please follow the SAP documentation on Uninstalling SAP Data Intelligence using the SLC Bridge (3.3).
Additionally, make sure to delete the sdi
project and datahub-system
as well:
# oc delete project sdi
Then delete datahub-system
project:
# oc delete project datahub-system
Optionally, one can also delete SDIObserver instance:
# oc delete SDIObserver sdiobserver-sample -n sdi-observer
When done, you may continue with a new installation round in the same or another namespace.
8.2. Quay Registry for SDI
Red Hat Quay Registry has been validated to host SAP Data Intelligence images. The Quay registry can run directly on the OpenShift cluster together with SDI, on another OpenShift cluster or standalone.
NOTE Red Hat Quay 3.6 or newer is compatible with SDI images.
Once Red Hat Quay is deployed according to the documentation, make sure to configure OpenShift cluster to trust the registry.
8.2.1. Quay namespaces, users and accounts preparations
-
Create a new organization. In this example, we will call the organization
sdi
.- This organization will host all the images needed by SLC Bridge, SAP DI and SAP DI operator.
-
As the Quay Superadmin, create a new user (e.g.
sdi_slcb
). Please note the credentials. The user will be used as a robot account by SLC Bridge and OpenShift (not by a human). So far, the regular Quay robot account cannot be used because the robot accounts cannot create repositories on push. -
Grant the
sdi_slcb
user at least theCreator
access to thesdi
organization either by adding the user to theowner's
team in "Teams and Membership" pane or by creating a new team in thesdi
organization called, e.g.,pushers
with theCreator
team role assigned and adding thesdi_slcb
user as a member. -
(optional) As the Superadmin, create another user for pipeline modeler (e.g.,
sdi_default_modeler
wheredefault
stands for the default tenant).- Advantages of having a separate registry namespace and users for each tenant's pipeline modeler are as follows:
- Images can be easily pruned on per-tenant basis. Once the SDI tenant is no longer needed, the corresponding Quay user can be deleted and its images will be automatically pruned from the registry and space recovered.
- Improved security. SDI tenant users cannot access images of other SDI tenants.
- This user will be used again as a robot account, similar to
sdi_slcb
. - For user's e-mail address, any fake address will do as long as it is unique among all Quay users.
- The name of the user is also the namespace where the images will be pushed to and pulled from.
- Make sure to note the credentials.
- The user must be able to pull from the
sdi
organization.
- Advantages of having a separate registry namespace and users for each tenant's pipeline modeler are as follows:
In order for the user to pull from sdi
organization, make sure to also perform the following.
- As the owner of the
sdi
organization, go to its "Teams and Membership" pane and create a new team (e.g.,pullers
) with theMember
Team Role. - Click on "Set permissions for pullers" and make sure the team can Read all the repositories that already exist in the
sdi
organization. - Click on the
puller
team, search forsdi_default_modeler
user and add him to the team. -
Go back to
Default Permissions
of thesdi
organization, click on "Create Default Permission" and add the "Read" permission to thepuller
team for repositories created byAnyone
. -
(optional) Repeat the previous step for any additional SDI tenant you are going to create.
8.2.2. Determining the Image Repository
The Image Repository Input Parameter is composed of <hostname>/<namespace>
.
- The registry
<hostname>
for Quay running on the OpenShift cluster can be determined on the Management host like this:
# oc get route --all-namespaces -o jsonpath='{range .items[*]}{.spec.host}{"\n"}{end}' \
-l quay-component=quay-app-route
An example output looks like:
quay.apps.cluster.example.com
If your local Quay registry runs outside of OpenShift cluster, you will need to determine its hostname by other means.
- The
<namespace>
is either the organization name or username. For thesdi
organization, the<namespace>
issdi
.
In this example, the resulting Image Repository parameter will be quay.apps.cluster.example.com/sdi
.
8.2.3. Importing Quay's CA certificate to OpenShift
If you haven't done it already, please make sure to make OpenShift cluster trust the Quay registry.
- If the Quay registry is running on the OpenShift cluster, obtain the
router-ca.crt
of the secret as documented in the SDI Registry Verification section. Otherwise, please fetch the self-signed CA certificate of your external Quay registry. - Follow section Configure OpenShift to trust container image registry to make the registry trusted.
8.2.4. Configuring additional SDI tenants
There are three steps that need to be performed for each new SDI tenant:
- Import the CA certificate for the registry via SDI Connection Manager if the CA certificate is self-signed.
- Create and import a vflow pull secret to OpenShift namespace.
- Create and import credential secret using SDI System Management and update the modeler secret.
In this example, we will operate with a newly created tenant blue
and we assume that new Quay registry user called blue_modeler
has been created.
8.2.4.1. Importing Quay's CA certificate to SAP DI
- Please follow step one from Importing Quay's CA Certificate to OpenShift to get the CA certificate locally as
router-ca.crt
. - Follow the Manage Certificates guide (3.3) / (3.2) / (3.1) to import the
router-ca.crt
via the SDI Connection Management.
8.2.4.2. Creating and importing vflow pull secret into OpenShift
This is needed only if a different Quay namespace is used for each tenant.
- Login to your Quay registry as the user
blue_modeler
. - Click on your user avatar in the upper right corner, go to "Account Settings" -> "User Settings" and there click on "Create Application Token". Let's use
blue_modeler_quay_token
as the token name. - Once the application token is generated, click on it and download the corresponding "Kubernetes Secret". In this example, the downloaded file is called
blue-modeler-quay-token-secret.yaml
. - On your Management host, import the secret into the
SDI_NAMESPACE
on your OpenShift cluster, e.g.:
# oc apply -n "${SDI_NAMESPACE:-sdi}" -f blue-modeler-quay-token-secret.yaml
- In SDI "System Management" of the
blue
tenant, go to the Applications tab, search forpull
, click on the Edit button and set "Modeler: Docker image pull secret for Modeler" to the name of the imported secret (e.g.,blue-modeler-quay-token-pull-secret
).
8.2.4.3. Importing credentials secret to SDI tenant
If you have imported the vflow pull secret into OpenShift cluster, you can turn the imported secret into the proper file format for SDI like this:
# secret=blue-modeler-quay-token-pull-secret
# oc get -o json -n "${SDI_NAMESPACE:-sdi}" "secret/$secret" | \
jq -r '.data[".dockerconfigjson"] | @base64d' | jq -r '.auths as $auths | $auths | keys |
map(. as $address | $auths[.].auth | @base64d | capture("^(?<username>[^:]+):(?<password>.+)$") |
{"address": $address, "username": .username, "password": .password})' | \
json2yaml | tee vsystem-registry-secret.txt
Otherwise, create the secret manually like this:
# cat >/tmp/vsystem-registry-secret.txt <<EOF
- username: "blue_modeler"
password: "CHANGEME"
address: "quay.apps.cluster.example.com"
EOF
NOTE The address must not contain any /<namespace>
suffix!
Import the secret using the SDI System Management by following the official Provide Access Credentials for a Password Protected Container Registry (3.3) / (3.2) / (3.1).
8.3. Configuring OpenShift to trust container image registry
If the registry's certificate is signed by a self-signed certificate authority, you must make OpenShift aware of it.
If the registry runs on the OpenShift cluster itself and is exposed via a reencrypt
or edge
route with the default TLS settings (no custom TLS certificates set), the CA certificate used is available in the secret router-ca
in the openshift-ingress-operator
namespace.
To make the registry available via such a trusted route, set the route's hostname into the registry
variable and execute the following code in bash:
# registry="quay.apps.morris.ocp.vslen"
# caBundle="$(oc get -n openshift-ingress-operator -o json secret/router-ca | \
jq -r '.data as $d | $d | keys[] | select(test("\\.(?:crt|pem)$")) | $d[.] | @base64d')"
# # determine the name of the CA configmap if it exists already
# cmName="$(oc get images.config.openshift.io/cluster -o json | \
jq -r '.spec.additionalTrustedCA.name // "trusted-registry-cabundles"')"
# if oc get -n openshift-config "cm/$cmName" 2>/dev/null; then
# configmap already exists -> just update it
oc get -o json -n openshift-config "cm/$cmName" | \
jq '.data["'"${registry//:/..}"'"] |= "'"$caBundle"'"' | \
oc replace -f - --force
else
# creating the configmap for the first time
oc create configmap -n openshift-config "$cmName" \
--from-literal="${registry//:/..}=$caBundle"
oc patch images.config.openshift.io cluster --type=merge \
-p '{"spec":{"additionalTrustedCA":{"name":"'"$cmName"'"}}}'
fi
If using a registry running outside of OpenShift or not secured by the default ingress CA certificate, take a look at the official guideline at Configuring a ConfigMap for the Image Registry Operator (4.14).
To verify that the CA certificate has been deployed, execute the following and check whether the supplied registry name appears among the file names in the output:
# oc rsh -n openshift-image-registry "$(oc get pods -n openshift-image-registry -l docker-registry=default | \
awk '/Running/ {print $1; exit}')" ls -1 /etc/pki/ca-trust/source/anchors
container-image-registry-sdi-observer.apps.boston.ocp.vslen
image-registry.openshift-image-registry.svc..5000
image-registry.openshift-image-registry.svc.cluster.local..5000
If this is not feasible, one can also mark the registry as insecure.
8.4. Configuring insecure registry
As a less secure alternative to the Configure OpenShift to trust container image registry, registry may also be marked as insecure, which poses a potential security risk. Please follow Configuring image settings (4.14) / (4.10) and add the registry to the .spec.registrySources.insecureRegistries
array. For example:
apiVersion: config.openshift.io/v1
kind: Image
metadata:
annotations:
release.openshift.io/create-only: "true"
name: cluster
spec:
registrySources:
insecureRegistries:
- local.image.registry:5000
NOTE: It may take a couple of minutes until the nodes are reconfigured. You can use the following commands to monitor the progress:
watch oc get machineconfigpool
watch oc get nodes
8.5. Running multiple SDI instances on a single OpenShift cluster
Two instances of SAP Data Intelligence running in parallel on a single OpenShift cluster have been validated. Running more instances is possible, but most likely requires an extra support statement from SAP.
Please consider the following before deploying more than one SDI instance to a cluster:
- Each SAP Data Intelligence instance must run in its own namespace/project.
- Each SAP Data Intelligence instance must use a different prefix or container image registry for the Pipeline Modeler. For example, the first instance can configure "Container Registry Settings for Pipeline Modeler" as
local.image.registry:5000/sdi30blue
and the second aslocal.image.registry:5000/sdi30green
. - It is recommended to dedicate particular nodes to each SDI instance.
- It is recommended to use network policy (4.14) SDN mode for completely granular network isolation configuration and improved security. Check network policy configuration (4.14) for further references and examples. This, however, cannot be changed post OpenShift installation.
- If running the production and test (aka blue-green) SDI deployments on a single OpenShift cluster, also keep in mind the following:
- There is no way to test an upgrade of OpenShift cluster before an SDI upgrade.
- The idle (non-productive) landscape should have the same network security as the live (productive) one.
To deploy a new SDI instance to OpenShift cluster, please repeat the steps from project setup, starting from point 6 with a new project name and continue with SDI Installation.
8.6. Installing remarshal utilities on RHEL
For a few example snippets throughout this guide, either yaml2json
or json2yaml
scripts are necessary.
They are provided by the remarshal project and shall be installed on the Management host in addition to jq
. On RHEL 8.2, one can install it this way:
# sudo dnf install -y python3-pip
# sudo pip3 install remarshal
8.7. Upgrading to the next minor release from the latest asynchronous release
If the OpenShift cluster is subscribed to the stable channel, its latest available micro release for the current minor release may not be upgradable to a newer minor release.
Consider the following example:
- The OpenShift cluster is version
4.13.24
. - The latest asynchronous release available in the stable-4.13 channel is
4.13.30
. - The latest stable 4.14 release is
4.14.15
(available instable-4.14
channel). - From the
4.13.24
micro release, one can upgrade to one of4.13.27
,4.13.28
,4.13.30
,4.14.13
or4.14.15
. - However, from
4.13.30
one cannot upgrade to any newer release because no upgrade path has been validated/provided yet in the stable channel.
Therefore, OpenShift cluster can get stuck on the 4.13 release if it is first upgraded to the latest asynchronous release, 4.13.30
, instead of being upgraded directly to one of the 4.14
minor releases. However, at the same time, the fast-4.14 channel contains the 4.14.16
release with an upgrade path from 4.13.30
. The 4.14.16
release appears in the stable-4.14
channel sooner or later after being introduced in the fast channel first.
To amend the situation without waiting for an upgrade path to appear in the stable channel:
- Temporarily switch to the fast-4.X channel.
- Perform the upgrade.
- Switch back to the stable-4.X channel.
- Continue performing upgrades to the latest micro release available in the stable-4.X channel.
8.8. HTTP proxy configuration
HTTP(S) proxy must be configured in different places. The corresponding no proxy settings are treated differently by different components:
- Management host
- OpenShift cluster
- SLC Bridge
- SAP Data Intelligence
The sections below assume the following:
- cluster's base domain is example.com
- cluster name is foo
, which means its API is listening at api.foo.example.com:6443
- the local proxy server is listening at http://proxy.example.com:3128
- Management host's hostname is jump.example.com, we should add its shortname (jump
) to the NO_PROXY
- the local network CIDR is 192.168.128.0/24
- the OpenShift's service network has the default range of 172.30.0.0/16
8.8.1. Configuring HTTP proxy on the Management host
Please export the proxy environment variables on your Management host according to your Linux distribution. For RHEL, please follow How to apply a system-wide proxy. For example, in BASH:
# sudo cp /dev/stdin /etc/profile.d/http_proxy.sh <<EOF
export http_proxy=http://proxy.example.com:3128
export https_proxy=http://proxy.example.com:3128
export no_proxy=localhost,127.0.0.1,jump,.example.com,192.168.128.0/24
EOF
# source /etc/profile.d/http_proxy.sh
Where the .example.com
is a wildcard pattern matching any subdomains like foo.example.com
.
8.8.2. Configuring HTTP proxy on the OpenShift cluster
Usually, OpenShift is configured to use the proxy during its installation.
But it is also possible to set/re-configure it ex-post.
An example configuration could look like this:
# oc get proxy/cluster -o json | jq '.spec'
{
"httpProxy": "http://proxy.example.com:3128",
"httpsProxy": "http://proxy.example.com:3128",
"noProxy": "192.168.128.0/24,jump,.local,.example.com",
"trustedCA": {
"name": "user-ca-bundle"
}
}
Please keep in mind that wildcard characters (e.g. *.example.com
) are not supported by OpenShift.
The complete no_proxy
list extended for container and service networks and additional service names is generated automatically and stored in the .status.noProxy
field of the proxy object:
# oc get proxy/cluster -o json | jq -r '.status.noProxy'
.cluster.local,.local,.example.com,.svc,10.128.0.0/14,127.0.0.1,172.30.0.0/16,192.168.128.0/24,api-int.foo.example.com,localhost,jump
8.8.3. Configuring HTTP proxy for the SLC Bridge
The SLC Bridge binary shall use the proxy settings from the environment on Management host configured earlier. This is important to allow SLCB to talk to the SAP image registry (proxied), the local image registry and the OpenShift API (not proxied).
During SLC Bridge's init phase, which deploys the bridge as a container on the OpenShift cluster, one must set proxy settings as well when prompted. Here are the example values:
# ./slcb init
...
***************************************************************************
* Choose whether you want to run the deployment in typical or expert mode *
***************************************************************************
1. Typical Mode
> 2. Expert Mode
Choose action <F12> for Back/<F1> for help
possible values [1,2]: 2
...
************************
* Proxy Settings *
************************
Configure Proxy Settings: y
Choose action <F12> for Back/<F1> for help
possible values [yes(y)/no(n)]: y
************************
* HTTPS Proxy *
************************
Enter the URL of the HTTPS Proxy to use
Choose action <F12> for Back/<F1> for help
HTTPS Proxy: http://proxy.example.com:3128
So far no surprise. For the `No Proxy` however, it is recommended to copy&append the `.status.noProxy` settings from the [OpenShift's proxy object](#apx-http-proxy-ocp).
************************
* Cluster No Proxy *
************************
Specify the NO_PROXY setting for the cluster. The value cannot contain white space and it must be comma-separated. You have to include the address range configured for the kubernetes cluster in this list (e.g. "10.240.0.0/20").
Choose action <F12> for Back/<F1> for help
Cluster No Proxy: 10.128.0.0/14,127.0.0.1,172.30.0.0/16,192.168.128.0/24,localhost,jump,169.254.169.254,sap-slcbridge,.local,.example.com,.svc,.internal
NOTE: You can use the following script to generate the value from OpenShift's proxy settings.
# bash <(curl -s https://raw.githubusercontent.com/redhat-sap/sap-data-intelligence/master/utils/get_no_proxy.sh) --slcb
Please make sure to append the --slcb
parameter.
8.8.4. Configuring HTTP Proxy for the SAP DI during its installation
During the SDI installation, one must choose "Advanced Installation" for the "Installation Type" in order to configure Proxy.
Then the following is an example of proxy settings:
**************************
* Cluster Proxy Settings *
**************************
Choose if you want to configure proxy settings on the cluster: y
Choose action <F12> for Back/<F1> for help
possible values [yes(y)/no(n)]: y
************************
* Cluster HTTP Proxy *
************************
Specify the HTTP_PROXY value for the cluster.
Choose action <F12> for Back/<F1> for help
HTTP_PROXY: http://proxy.example.com:3128
************************
* Cluster HTTPS Proxy *
************************
Specify the HTTPS_PROXY value for the cluster.
Choose action <F12> for Back/<F1> for help
HTTPS_PROXY: http://proxy.ocpoff.vslen:3128
************************
* Cluster No Proxy *
************************
Specify the NO_PROXY value for the cluster. NO_PROXY value cannot contain white spaces and it must be comma-separated.
Choose action <F12> for Back/<F1> for help
NO_PROXY: 10.0.0.0/16,10.128.0.0/14,127.0.0.1,172.30.0.0/16,192.168.0.0/16,192.168.128.2,localhost,jump,169.254.169.254,auditlog,datalake,diagnostics-prometheus-pushgateway,hana-service,storagegateway,uaa,vora-consul,vora-dlog,vora-prometheus-pushgateway,vsystem,vsystem-internal,*.local,*.example.com,*.svc,*.internal
NOTE:The value can be generated using the following script:
# bash <(curl -s https://raw.githubusercontent.com/redhat-sap/sap-data-intelligence/master/utils/get_no_proxy.sh)
# # to see the usage and options, append `--help`
# bash <(curl -s https://raw.githubusercontent.com/redhat-sap/sap-data-intelligence/master/utils/get_no_proxy.sh) --help
When setting the No Proxy
, please keep in mind the following:
- The wildcard domains must contain wildcard characters. On the contrary, OpenShift's proxy settings must not contain wildcard characters.
- As of SLC Bridge 1.1.72,
NO_PROXY
must not start with a wildcard domain. Please put the wildcard domains at the end ofNO_PROXY
. - In addition to the OpenShift Proxy's
.status.noProxy
values, the list should also include these service names:vora-consul,hana-service,uaa,auditlog,vora-dlog,vsystem-internal,vsystem,vora-prometheus-pushgateway,diagnostics-prometheus-pushgateway,storagegateway,datalake
.
8.8.5. Configuring HTTP Proxy after the SAP DI installation
- Login to the
system
tenant as aclusterAdmin
and openSystem Management
. - Click on
Cluster
and then click onTenants
. - For each tenant, click on the tenant row.
- Click on "View Application Configuration and Secrets".
- Search for
PROXY
and click on the Edit button. - Edit the values as needed. Feel free to use the
get_no_proxy.sh
script above to generate theNo proxy
value. - Click the
Update
button. - (If dealing with a
system
tenant, please skip this step until the very end). Go back to the tenant overview. This time, click on "Delete all Instances". Please note that this will cause a slight downtime for the tenant's current users. - Repeat for other tenants from step 3.
- Execute step 8 for the
system
tenant as well.
8.9. Enabling GPU for SDI on OCP
To enable the GPU usage for SDI on OCP, please refer to GPU enablement for SDI on OCP.
9. Troubleshooting
Please refer to Troubleshooting Guide for SAP Data Intelligence 3 on OpenShift Container Platform 4, for the detailed troubleshooting guide.
Comments