SSSD user logins fail due to failed TGT validation

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 8
  • SSSD
  • Active Directory

Issue

  • Unable to login with SSSD configured using the AD provider
  • Able to resolve AD Trust users in IDM but logins fail
  • SSSD /var/log/sssd/krb5_child errors when attempting logins
[[sssd[krb5_child]]] [validate_tgt] (0x0020): TGT failed verification using key for [host/idmsystem.example.com@EXAMPLE.COM]
[[sssd[krb5_child]]] [get_and_save_tgt] (0x0020): [-1765328377][Server not found in Kerberos database]
[[sssd[krb5_child]]] [map_krb5_error] (0x0020): [-1765328377][Server not found in Kerberos database]
  • Active Directory is rejecting trusted host principals:
[krb5_child] [get_and_save_tgt] (0x0020): [-1765328372][KDC policy rejects request]
[krb5_child] [map_krb5_error] (0x0020): [-1765328372][KDC policy rejects request]

Resolution

When kerberos validation fails, users will be denied from logging in during SSSD PAM Authentication. There are a number reasons that kerberos validation could fail, some known reasons are mentioned below:

1. Invalid or extra entries in the /etc/krb5.keytab:

As explained in the sssd-krb5 man page, the keytab is checked for entries sequentially for a matching realm, if no entry matches the realm then the last entry is used.

Check for duplicate or invalid entries in the following output:

# klist -ke

2. Missing SPN values in AD:

With the AD provider, an incomplete 'Join' operation could cause missing servicePrincipalName attributes on the RHEL machine's Active Directory Computer Object LDAP Entry. Check that the servicePrincipalName attribute within the Computer Object LDAP Entry match with the klist -k output on the SSSD system.

3. Unreachable AD Domain Controllers in the trust path:

Intermediary Domain Controllers may need to be contacted during validation if there is no direct trust path from the SSSD host AD domain to the user's domain. If there are Active Directory Domain Controllers which are unreachable by the SSSD Client, make sure to remove these from the DNS Zone that the client is using so that it does not try to contact them.

4. Leftover Trust in Active Directory:

Validation fails when IDM has an established trust with the AD Forest Root domain but a stale trust exists in Active Directory from a previous IDM external trust creation.

Check the Trusts tab in the Active Directory Domains and Trusts console to ensure there is only a trust between the AD domain that was provided as an argument to the ipa trust-add command and IDM.

5. Incorrect capaths configuration (Fixed in RHEL 7.3 sssd-1.14.0-43):

In certain cases environments, SSSD would create an incorrect krb5 configuration snippet for capaths configuration - Upstream Ticket

6. Active Directory policy rejects the use of trusted Host Principals:

In some Active Directory environments, the policies set within Active Directory will not allow the use of Trusted Host Principals. The "Allowed to Authenticate" rule is required to be enabled in order for authentication of Active Directory users to the trusted machine machine to function. See this Microsoft article for more information:
* https://support.microsoft.com/en-us/topic/c4cafc63-3da3-a81e-eb21-915f61af9ca3

Note:

  • The krb5_validate option can be disabled temporarily for testing purpose to ensure that kerberos validation is causing the login problems experienced. NOTE: Not Recommended for Security reasons explained in the Root Cause section

      krb5_validate = false

  • Setting ad_maximum_machine_account_password_age=0 in the [domain/...] section of sssd.conf also, may work as a workaround.

  • By adding udp_preference_limit = 0 to the [libdefaults] section of /etc/krb5.conf, libkrb5 would use TCP by default which sometimes resolves the issue.

Root Cause

  • Kerberos validation is enabled by default to ensure an attacker does not tamper with TGT data,. Disabling validation can allow an unauthorized user to gain privileges from the KDC using a man-in-the-middle attack replacing or adding to expected user data. The following Microsoft PAC Validation article goes into detail about PAC validation, which is another variant of TGT validation.

Diagnostic Steps

  • Review the /var/log/sssd/sssd_<domain>.log for similar errors and check the principal used to get the TGT
[sssd[be[example.com]]] [child_handler_setup] (0x2000): Setting up signal handler up for pid [4138]
[sssd[be[example.com]]] [child_handler_setup] (0x2000): Signal handler set up for pid [4138]
[sssd[be[example.com]]] [write_pipe_handler] (0x0400): All data has been sent!
[sssd[be[example.com]]] [child_sig_handler] (0x1000): Waiting for child [4138].
[sssd[be[example.com]]] [child_sig_handler] (0x0100): child [4138] finished successfully.
[sssd[be[example.com]]] [read_pipe_handler] (0x0400): EOF received, client finished
[sssd[be[example.com]]] [parse_krb5_child_response] (0x1000): child response [1432158209][6][8].
[sssd[be[example.com]]] [check_wait_queue] (0x1000): Wait queue for user [ad_user@adcorp.test] is empty.
[sssd[be[example.com]]] [krb5_auth_queue_done] (0x1000): krb5_auth_queue request [0x2149e10] done.
[sssd[be[example.com]]] [be_pam_handler_callback] (0x0100): Backend returned: (0, 4, <NULL>) [Success]
[sssd[be[example.com]]] [be_pam_handler_callback] (0x0100): Sending result [4][example.com]
[sssd[be[example.com]]] [be_pam_handler_callback] (0x0100): Sent result [4][example.com]
  • Try testing the kerberos validation outside of SSSD
# kdestroy -A
# kinit ad_user@ADCORP.TEST
# KRB5_TRACE=/dev/stdout kvno host/$(hostname)@EXAMPLE.COM

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments