Single user from LDAP or OIDC cannot login with error "unexpected response: 500" in OCP

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 3
    • 4
  • LDAP
  • OpenID

Issue

  • Some users from OpenID are not able to sign into the Openshift cluster.
  • After recreating LDAP secret to the Oauth pods, one single user still can not login.
  • Error message "Internal error occurred: unexpected response: 500" is shown when trying to login via oc command.
  • Error message "An authentication error occurred" is shown by few users when trying to login via the OpenShift web console.

Resolution

Identify the identity associated to the user and delete it:

$ oc get identities
$ oc get identity [identity_name] -o yaml
[...]
$ oc delete identity [identity_name]

OpenShift should be able to resync automatically when trying to login again. In some cases, it could be needed to delete the user after deleting the identity. Before doing that, backup the user:

$ oc get user [user_name] -o yaml > [backup-user_name].yaml

Root Cause

The identity associated with the user contains incorrect data and causes the login to fail for that user.

Diagnostic Steps

Try to login with oc command using --log-level=10:

# oc login --log-level=10
Authentication required for https://api.openshift.example.local:6443 (openshift)
Username: my_user@example.local
Password:
I1028 09:14:18.470402   13630 round_trippers.go:423] curl -k -v -XGET  -H "Authorization: Basic QHNzYnRwISEh" -H "X-Csrf-Token: 1" 'https://oauth-openshift.apps.openshift.example.local/oauth/authorize?client_id=openshift-challenging-client&code_challenge=r8&code_challenge_method=S256&redirect_uri=https%3A%2F%2Foauth-openshift.apps.openshift.example.local%2Foauth%2Ftoken%2Fimplicit&response_type=code'
I1028 09:14:18.510571   13630 round_trippers.go:443] GET https://oauth-openshift.apps.openshift.example.local/oauth/authorize?client_id=openshift-challenging-client&code_challenge=3pUwBsHlCyY1_i-lizYK6oNz04C5mdYhgbR9-W17yr8&code_challenge_method=S256&redirect_uri=https%3A%2F%2Foauth-openshift.apps.openshift.example.local%2Foauth%2Ftoken%2Fimplicit&response_type=code 500 Internal Server Error in 40 milliseconds
I1028 09:14:18.510602   13630 round_trippers.go:449] Response Headers:
I1028 09:14:18.510610   13630 round_trippers.go:452]     Expires: 0
I1028 09:14:18.510616   13630 round_trippers.go:452]     Referrer-Policy: strict-origin-when-cross-origin
I1028 09:14:18.510623   13630 round_trippers.go:452]     X-Dns-Prefetch-Control: off
I1028 09:14:18.510628   13630 round_trippers.go:452]     X-Frame-Options: DENY
I1028 09:14:18.510633   13630 round_trippers.go:452]     Date: Wed, 28 Oct 2020 12:14:18 GMT
I1028 09:14:18.510638   13630 round_trippers.go:452]     Cache-Control: no-cache, no-store, max-age=0, must-revalidate
I1028 09:14:18.510645   13630 round_trippers.go:452]     Pragma: no-cache
I1028 09:14:18.510650   13630 round_trippers.go:452]     X-Content-Type-Options: nosniff
I1028 09:14:18.510655   13630 round_trippers.go:452]     X-Xss-Protection: 1; mode=block
I1028 09:14:18.510668   13630 round_trippers.go:452]     Content-Length: 65
I1028 09:14:18.510673   13630 round_trippers.go:452]     Content-Type: text/plain; charset=utf-8
I1028 09:14:18.511642   13630 round_trippers.go:423] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: oc/v0.0.0 (linux/amd64) kubernetes/$Format" 'https://api.openshift.example.local:6443/api/v1/namespaces/openshift/configmaps/motd'
I1028 09:14:18.513482   13630 round_trippers.go:443] GET https://api.openshift.example.local:6443/api/v1/namespaces/openshift/configmaps/motd 403 Forbidden in 1 milliseconds
I1028 09:14:18.513499   13630 round_trippers.go:449] Response Headers:
I1028 09:14:18.513505   13630 round_trippers.go:452]     Audit-Id: 89bfff8f-0f37-4d05-9483-b30823b1197c
I1028 09:14:18.513510   13630 round_trippers.go:452]     Content-Type: application/json
I1028 09:14:18.513515   13630 round_trippers.go:452]     X-Content-Type-Options: nosniff
I1028 09:14:18.513520   13630 round_trippers.go:452]     Content-Length: 303
I1028 09:14:18.513525   13630 round_trippers.go:452]     Date: Wed, 28 Oct 2020 12:14:39 GMT
I1028 09:14:18.513566   13630 request.go:968] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"motd\" is forbidden: User \"system:anonymous\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"openshift\"","reason":"Forbidden","details":{"name":"motd","kind":"configmaps"},"code":403}
I1028 09:14:18.514274   13630 helpers.go:199] server response object: [{
  "metadata": {},
  "status": "Failure",
  "message": "Internal error occurred: unexpected response: 500",
  "reason": "InternalError",
  "details": {
    "causes": [
      {
        "message": "unexpected response: 500"
      }
    ]
  },
  "code": 500

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments