OpenShift 4.20 installation through MCE Assisted Service fails with 'crun: systemd failed to install eBPF device filter'

Solution Verified - Updated -

Issue

An OpenShift 4.20 cluster installation provisioned through the Assisted Service provided by multicluster engine for Kubernetes (MCE) fails during the discovery/live environment bootstrap phase.

The affected host reports the following crun error:

Error: OCI runtime error: crun: systemd failed to install eBPF device filter on cgroup `/sys/fs/cgroup/machine.slice/libpod-<container-id>.scope`

The failure can cause crio-configure.service to exit with status 126:

crio-configure.service: Main process exited, code=exited, status=126/n/a
crio-configure.service: Failed with result 'exit-code'.
Failed to start Configure CRI-O to use the pause image.

bootkube.service then fails because its dependency did not complete:

Dependency failed for Bootstrap a Kubernetes cluster.
bootkube.service: Job bootkube.service/start failed with result 'dependency'.

Affected hosts can also report SELinux/eBPF errors such as:

avc: denied { prog_run } for pid=1 comm=systemd scontext=system_u:system_r:init_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=bpf permissive=0

and:

bpf-foreign: Failed to create foreign BPF program: Permission denied
bpf-foreign: Failed to prepare foreign BPF hashmap: Permission denied

The SELinux AVC and bpf-foreign messages might not be present in every affected environment. The common bootstrap symptom is the crun eBPF device-filter failure followed by the crio-configure.service and bootkube.service failure chain.

Environment

  • Red Hat OpenShift Container Platform 4.20
  • multicluster engine for Kubernetes 2.10 with Assisted Service enabled
  • x86_64 architecture

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content