[StackRox] Troubleshooting why the RHACS (StackRox) Collectors are in a CrashLoopBackOff state after upgrading a AKS Cluster to v1.18.14

Solution Unverified - Updated -

Issue

  • Upon upgrading an AKS cluster to 1.18.14, the following components are reporting error messages similar to the ones demonstrated below;

Collector

  • Version 3.0.13 Collectors are in a CrashLoopBackOff state and the log output is reporting the following error message;
2021-03-03T14:31:32.372915296Z curl: (7) Failed to connect to sensor.stackrox port 443: Connection refused
2021-03-03T14:31:34.388864035Z curl: (7) Failed to connect to sensor.stackrox port 443: Connection refused
2021-03-03T14:31:36.404901575Z curl: (7) Failed to connect to sensor.stackrox port 443: Connection refused
2021-03-03T14:31:38.420885414Z curl: (7) Failed to connect to sensor.stackrox port 443: Connection refused
2021-03-03T14:31:40.436909253Z curl: (7) Failed to connect to sensor.stackrox port 443: Connection refused
2021-03-03T14:31:40.441229256Z Failed to download kernel module.
2021-03-03T14:31:40.441463156Z All attempts to download the kernel module have failed.
2021-03-03T14:31:40.441582256Z The kernel module may not have been compiled for version 5.4.0-1039-azure.
2021-03-03T14:31:40.441717156Z Error: Failed to find kernel module for kernel version 5.4.0-1039-azure.
2021-03-03T14:31:40.441842956Z
2021-03-03T14:31:40.441933756Z Please provide this complete error message to StackRox support.
2021-03-03T14:31:40.442036156Z This program will now exit and retry when it is next restarted.
2021-03-03T14:31:40.442180256Z
2021-03-03T14:31:40.539128611Z HTTP Status Code 000

Sensor

  • Version 3.0.50.1 Sensor log output reports the following error message;
common/sensor: 2021/03/03 16:55:51.640781 sensor.go:270: Error: Sensor reported an error: receiving initial cluster config: rpc error: code = PermissionDenied desc = not authorized: no identity in context
main: 2021/03/03 16:55:51.640839 main.go:68: Fatal: Sensor exited with error: receiving initial cluster config: rpc error: code = PermissionDenied desc = not authorized: no identity in context

Central

  • Version 3.0.50.1 Central log output reports the following error message;
pkg/grpc/authn: 2021/03/03 16:39:27.346977 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.692312192s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:39:27.500219 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.744584045s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:44:32.386261 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.689826684s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:44:32.617708 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.742961089s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:49:36.244335 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.72795163s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:49:36.458158 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.767501771s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:54:49.466061 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.714695046s > 1m0s detected
pkg/grpc/authn: 2021/03/03 16:54:49.695463 interceptor.go:22: Error: Error extracting identity: could not parse service cert token: time difference 1m1.757255829s > 1m0s detected

Environment

  • StackRox Version - 3.0.50.1 or earlier.
  • Orchestrator - Azure Kubernetes Service (AKS) - v1.18.14.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content