Chapter 7. Security Profiles Operator

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators → OperatorHub.
2. Search for the Security Profiles Operator, then click Install.
3. Keep the default selection of Installation mode and namespace to ensure that the Operator will be installed to the openshift-security-profiles namespace.
4. Click Install.

1. Navigate to the Operators → Installed Operators page.
2. Check that the Security Profiles Operator is installed in the openshift-security-profiles namespace and its status is Succeeded.

1. Navigate to the Operators → Installed Operators page and inspect the Status column for any errors or failures.
2. Navigate to the Workloads → Pods page and check the logs in any pods in the openshift-security-profiles project that are reporting issues.

Procedure

1. Define a Namespace object:

   Example namespace-object.yaml

   apiVersion: v1
   kind: Namespace
   metadata:
       name: openshift-security-profiles
   labels:
     openshift.io/cluster-monitoring: "true"

2. Create the Namespace object:

   $ oc create -f namespace-object.yaml

3. Define an OperatorGroup object:

   Example operator-group-object.yaml

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: security-profiles-operator
     namespace: openshift-security-profiles

4. Create the OperatorGroup object:

   $ oc create -f operator-group-object.yaml

5. Define a Subscription object:

   Example subscription-object.yaml

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: security-profiles-operator-sub
     namespace: openshift-security-profiles
   spec:
     channel: release-alpha-rhel-8
     installPlanApproval: Automatic
     name: security-profiles-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace

6. Create the Subscription object:

   $ oc create -f subscription-object.yaml

Verification

1. Verify the installation succeeded by inspecting the following CSV file:

   $ oc get csv -n openshift-security-profiles

2. Verify that the Security Profiles Operator is operational by running the following command:

   $ oc get deploy -n openshift-security-profiles

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

2. Create the SeccompProfile object:

   apiVersion: security-profiles-operator.x-k8s.io/v1beta1
   kind: SeccompProfile
   metadata:
     namespace: my-namespace
     name: profile1
   spec:
     defaultAction: SCMP_ACT_LOG

Procedure

1. Create a pod object that defines a securityContext:

   apiVersion: v1
   kind: Pod
   metadata:
     name: test-pod
   spec:
     securityContext:
       seccompProfile:
         type: Localhost
         localhostProfile: operator/my-namespace/profile1.json
     containers:
       - name: test-container
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21

2. View the profile path of the seccompProfile.localhostProfile attribute by running the following command:

   $ oc -n my-namespace get seccompprofile profile1 --output wide

   Example output

   NAME       STATUS     AGE   SECCOMPPROFILE.LOCALHOSTPROFILE
   profile1   Installed  14s   operator/my-namespace/profile1.json

3. View the path to the localhost profile by running the following command:

   $ oc get sp profile1 --output=jsonpath='{.status.localhostProfile}'

   Example output

   operator/my-namespace/profile1.json

4. Apply the localhostProfile output to the patch file:

   spec:
     template:
       spec:
         securityContext:
           seccompProfile:
             type: Localhost
             localhostProfile: operator/my-namespace/profile1.json

5. Apply the profile to any other workload, such as a Deployment object, by running the following command:

   $ oc -n my-namespace patch deployment myapp --patch-file patch.yaml --type=merge

   Example output

   deployment.apps/myapp patched

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

2. Label the namespace with enable-recording=true by running the following command:

   $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true

3. Create a ProfileRecording object containing a recorder: logs variable:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
   kind: ProfileRecording
   metadata:
     namespace: my-namespace
     name: test-recording
   spec:
     kind: SeccompProfile
     recorder: logs
     podSelector:
       matchLabels:
         app: my-app

4. Create a workload to record:

   apiVersion: v1
   kind: Pod
   metadata:
     namespace: my-namespace
     name: my-pod
     labels:
       app: my-app
   spec:
     containers:
       - name: nginx
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         ports:
           - containerPort: 8080
       - name: redis
         image: quay.io/security-profiles-operator/redis:6.2.1

5. Confirm the pod is in a Running state by entering the following command:

   $ oc -n my-namespace get pods

   Example output

   NAME     READY   STATUS    RESTARTS   AGE
   my-pod   2/2     Running   0          18s

6. Confirm the enricher indicates that it receives audit logs for those containers:

   $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher

   Example output

   I0523 14:19:08.747313  430694 enricher.go:445] log-enricher "msg"="audit" "container"="redis" "executable"="/usr/local/bin/redis-server" "namespace"="my-namespace" "node"="xiyuan-23-5g2q9-worker-eastus2-6rpgf" "pid"=656802 "pod"="my-pod" "syscallID"=0 "syscallName"="read" "timestamp"="1684851548.745:207179" "type"="seccomp"

Verification

1. Remove the pod:

   $ oc -n my-namepace delete pod my-pod

2. Confirm the Security Profiles Operator reconciles the two seccomp profiles:

   $ oc get seccompprofiles -lspo.x-k8s.io/recording-id=test-recording -n my-namespace

   Example output for seccompprofile

   NAME                   STATUS      AGE
   test-recording-nginx   Installed   2m48s
   test-recording-redis   Installed   2m48s

Procedure

1. Create a project by running the following command:

   $ oc new-project nginx-deploy

2. Create a policy that can be used with a non-privileged workload by creating the following SelinuxProfile object:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha2
   kind: SelinuxProfile
   metadata:
     name: nginx-secure
     namespace: nginx-deploy
   spec:
     allow:
       '@self':
         tcp_socket:
         - listen
       http_cache_port_t:
         tcp_socket:
         - name_bind
       node_t:
         tcp_socket:
         - node_bind
     inherit:
     - kind: System
       name: container

3. Wait for selinuxd to install the policy by running the following command:

   $ oc wait --for=condition=ready -n nginx-deploy selinuxprofile nginx-secure

   Example output

   selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure condition met

   The policies are placed into an emptyDir in the container owned by the Security Profiles Operator. The policies are saved in Common Intermediate Language (CIL) format in /etc/selinux.d/<name>_<namespace>.cil.

4. Access the pod by running the following command:

   $ oc -n openshift-security-profiles rsh -c selinuxd ds/spod

Verification

1. View the file contents with cat by running the following command:

   $ cat /etc/selinux.d/nginx-secure_nginx-deploy.cil

   Example output

   (block nginx-secure_nginx-deploy
   (blockinherit container)
   (allow process nginx-secure_nginx-deploy.process ( tcp_socket ( listen )))
   (allow process http_cache_port_t ( tcp_socket ( name_bind )))
   (allow process node_t ( tcp_socket ( node_bind )))
   )

2. Verify that a policy has been installed by running the following command:

   $ semodule -l | grep nginx-secure

   Example output

   nginx-secure_nginx-deploy

Procedure

1. Apply the scc.podSecurityLabelSync=false label to the nginx-deploy namespace by running the following command:

   $ oc label ns nginx-deploy security.openshift.io/scc.podSecurityLabelSync=false

2. Apply the privileged label to the nginx-deploy namespace by running the following command:

   $ oc label ns nginx-deploy --overwrite=true pod-security.kubernetes.io/enforce=privileged

3. Obtain the SELinux profile usage string by running the following command:

   $ oc get selinuxprofile.security-profiles-operator.x-k8s.io/nginx-secure -n nginx-deploy -ojsonpath='{.status.usage}'

   Example output

   nginx-secure_nginx-deploy.process

4. Apply the output string in the workload manifest in the .spec.containers[].securityContext.seLinuxOptions attribute:

   apiVersion: v1
   kind: Pod
   metadata:
     name: nginx-secure
     namespace: nginx-deploy
   spec:
     containers:
       - image: nginxinc/nginx-unprivileged:1.21
         name: nginx
         securityContext:
           seLinuxOptions:
             # NOTE: This uses an appropriate SELinux type
             type: nginx-secure_nginx-deploy.process

   Important

   The SELinux type must exist before creating the workload.

Procedure

1. Create a project by running the following command:

   $ oc new-project my-namespace

2. Label the namespace with enable-recording=true by running the following command:

   $ oc label ns my-namespace spo.x-k8s.io/enable-recording=true

3. Create a ProfileRecording object containing a recorder: logs variable:

   apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
   kind: ProfileRecording
   metadata:
     namespace: my-namespace
     name: test-recording
   spec:
     kind: SelinuxProfile
     recorder: logs
     podSelector:
       matchLabels:
         app: my-app

4. Create a workload to record:

   apiVersion: v1
   kind: Pod
   metadata:
     namespace: my-namespace
     name: my-pod
     labels:
       app: my-app
   spec:
     containers:
       - name: nginx
         image: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
         ports:
           - containerPort: 8080
       - name: redis
         image: quay.io/security-profiles-operator/redis:6.2.1

5. Confirm the pod is in a Running state by entering the following command:

   $ oc -n my-namespace get pods

   Example output

   NAME     READY   STATUS    RESTARTS   AGE
   my-pod   2/2     Running   0          18s

6. Confirm the enricher indicates that it receives audit logs for those containers:

   $ oc -n openshift-security-profiles logs --since=1m --selector name=spod -c log-enricher

   Example output

   I0517 13:55:36.383187  348295 enricher.go:376] log-enricher "msg"="audit" "container"="redis" "namespace"="my-namespace" "node"="ip-10-0-189-53.us-east-2.compute.internal" "perm"="name_bind" "pod"="my-pod" "profile"="test-recording_redis_6kmrb_1684331729" "scontext"="system_u:system_r:selinuxrecording.process:s0:c4,c27" "tclass"="tcp_socket" "tcontext"="system_u:object_r:redis_port_t:s0" "timestamp"="1684331735.105:273965" "type"="selinux"

Verification

1. Remove the pod:

   $ oc -n my-namepace delete pod my-pod

2. Confirm the Security Profiles Operator reconciles the two SELinux profiles:

   $ oc get selinuxprofiles -lspo.x-k8s.io/recording-id=test-recording -n my-namespace

   Example output for selinuxprofile

   NAME                   USAGE                                       STATE
   test-recording-nginx   test-recording-nginx_my-namespace.process   Installed
   test-recording-redis   test-recording-redis_my-namespace.process   Installed

Procedure

1. Enable memory optimization by running the following command:

   $ oc -n openshift-security-profiles patch spod spod --type=merge -p '{"spec":{"enableMemoryOptimization":true}}'

2. To record a security profile for a pod, the pod must be labeled with spo.x-k8s.io/enable-recording: "true":

   apiVersion: v1
   kind: Pod
   metadata:
     name: my-recording-pod
     labels:
       spo.x-k8s.io/enable-recording: "true"

Procedure

1. To view the status of the metrics service, run the following command:

   $ oc get svc/metrics -n openshift-security-profiles

   Example output

   NAME      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
   metrics   ClusterIP   10.0.0.228   <none>        443/TCP   43s

2. To retrieve the metrics, query the service endpoint using the default ServiceAccount token in the openshift-security-profiles namespace by running the following command:

   $ oc run --rm -i --restart=Never --image=registry.fedoraproject.org/fedora-minimal:latest \
       -n openshift-security-profiles metrics-test -- bash -c \
       'curl -ks -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://metrics.openshift-security-profiles/metrics-spod'

   Example output

   # HELP security_profiles_operator_seccomp_profile_total Counter about seccomp profile operations.
   # TYPE security_profiles_operator_seccomp_profile_total counter
   security_profiles_operator_seccomp_profile_total{operation="delete"} 1
   security_profiles_operator_seccomp_profile_total{operation="update"} 2

3. To retrieve metrics from a different namespace, link the ServiceAccount to the spo-metrics-client ClusterRoleBinding by running the following command:

   $ oc get clusterrolebinding spo-metrics-client -o wide

   Example output

   NAME                 ROLE                             AGE   USERS   GROUPS   SERVICEACCOUNTS
   spo-metrics-client   ClusterRole/spo-metrics-client   35m                    openshift-security-profiles/default

Procedure

1. Patch the spod configuration to enable the log enricher by running the following command:

   $ oc -n openshift-security-profiles patch spod spod \
       --type=merge -p '{"spec":{"enableLogEnricher":true}}'

   Example output

   securityprofilesoperatordaemon.security-profiles-operator.x-k8s.io/spod patched

   Note

   The Security Profiles Operator will re-deploy the spod daemon set automatically.

2. View the audit logs by running the following command:

   $ oc -n openshift-security-profiles logs -f ds/spod log-enricher

   Example output

   I0623 12:51:04.257814 1854764 deleg.go:130] setup "msg"="starting component: log-enricher"  "buildDate"="1980-01-01T00:00:00Z" "compiler"="gc" "gitCommit"="unknown" "gitTreeState"="clean" "goVersion"="go1.16.2" "platform"="linux/amd64" "version"="0.4.0-dev"
   I0623 12:51:04.257890 1854764 enricher.go:44] log-enricher "msg"="Starting log-enricher on node: 127.0.0.1"
   I0623 12:51:04.257898 1854764 enricher.go:46] log-enricher "msg"="Connecting to local GRPC server"
   I0623 12:51:04.258061 1854764 enricher.go:69] log-enricher "msg"="Reading from file /var/log/audit/audit.log"
   2021/06/23 12:51:04 Seeked /var/log/audit/audit.log - &{Offset:0 Whence:2}

Procedure

1. Set the recording.spo.io webhook configuration to record only pods labeled with spo-record=true by creating the following patch file:

   spec:
     webhookOptions:
       - name: recording.spo.io
         objectSelector:
           matchExpressions:
             - key: spo-record
               operator: In
               values:
                 - "true"

2. Patch the spod/spod instance by running the following command:

   $ oc -n openshift-security-profiles patch spod \
       spod -p $(cat /tmp/spod-wh.patch) --type=merge

3. To view the resulting MutatingWebhookConfiguration object, run the following command:

   $ oc get MutatingWebhookConfiguration \
       spo-mutating-webhook-configuration -oyaml

Procedure

1. Confirm that the profile is reconciled by running the following command:

   $ oc -n openshift-security-profiles logs openshift-security-profiles-<id>

   Example 7.2. Example output

   I1019 19:34:14.942464       1 main.go:90] setup "msg"="starting openshift-security-profiles"  "buildDate"="2020-10-19T19:31:24Z" "compiler"="gc" "gitCommit"="a3ef0e1ea6405092268c18f240b62015c247dd9d" "gitTreeState"="dirty" "goVersion"="go1.15.1" "platform"="linux/amd64" "version"="0.2.0-dev"
   I1019 19:34:15.348389       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"=":8080"
   I1019 19:34:15.349076       1 main.go:126] setup "msg"="starting manager"
   I1019 19:34:15.349449       1 internal.go:391] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
   I1019 19:34:15.350201       1 controller.go:142] controller "msg"="Starting EventSource" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"defaultAction":""}}}
   I1019 19:34:15.450674       1 controller.go:149] controller "msg"="Starting Controller" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile"
   I1019 19:34:15.450757       1 controller.go:176] controller "msg"="Starting workers" "controller"="profile" "reconcilerGroup"="security-profiles-operator.x-k8s.io" "reconcilerKind"="SeccompProfile" "worker count"=1
   I1019 19:34:15.453102       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="nginx-1.19.1" "name"="nginx-1.19.1" "resource version"="728"
   I1019 19:34:15.453618       1 profile.go:148] profile "msg"="Reconciled profile from SeccompProfile" "namespace"="openshift-security-profiles" "profile"="openshift-security-profiles" "name"="openshift-security-profiles" "resource version"="729"

2. Confirm that the seccomp profiles are saved into the correct path by running the following command:

   $ oc exec -t -n openshift-security-profiles openshift-security-profiles-<id> \
       -- ls /var/lib/kubelet/seccomp/operator/my-namespace/my-workload

   Example output

   profile-block.json
   profile-complain.json

1. Navigate to the Operators → Installed Operators page.
2. Delete all seccomp profiles, SELinux profiles, and webhook configurations.
3. Switch to the Administration → Operators → Installed Operators page.
4. Click the Options menu on the Security Profiles Operator entry and select Uninstall Operator.
5. Switch to the Home → Projects page.
6. Search for security profiles.

7. Click the Options menu next to the openshift-security-profiles project, and select Delete Project.

   1. Confirm the deletion by typing openshift-security-profiles in the dialog box, and click Delete.

8. Delete the MutatingWebhookConfiguration object by running the following command:

   $ oc delete MutatingWebhookConfiguration spo-mutating-webhook-configuration