After sssd upgrade, useradd and usermod fail to flush the sssd cache with "DB version too old for domain implicit_files!".

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 9
  • SSSD
  • shadow-utils

Issue

  • After sssd upgrade, useradd and usermod fail to flush the sssd cache with "DB version too old for domain implicit_files!".

  • Sample reproducer:

    # useradd testuser0
# systemctl is-enabled sssd.service
# systemctl disable sssd.service
# systemctl stop sssd.service
# yum update sssd

  • Sample outputs of a sssd upgrade system from sssd-2.3.0-9.el8(RHEL8.3) to sssd-2.5.2-2.el8(RHEL8.5):

    # useradd testuser1
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
useradd: sss_cache exited with status 70
useradd: Failed to flush the sssd cache.
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
useradd: sss_cache exited with status 70
useradd: Failed to flush the sssd cache.

# id testuser1
uid=1003(testuser1) gid=1003(testuser1) groups=1003(testuser1)

# usermod --comment test testuser1
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
usermod: sss_cache exited with status 70
usermod: Failed to flush the sssd cache.
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
usermod: sss_cache exited with status 70
usermod: Failed to flush the sssd cache.

# getent passwd testuser1
testuser1:x:1003:1003:test:/home/testuser1:/bin/bash

Resolution

  • There is a bug reported for this and it is fixed in RHEL10:
    SSSD DB version too old after upgrade

  • For RHEL 9, the fix is included in RHBA-2024:11243.

  • Issue won't be fixed in RHEL 8, as a workaround, fix the issue just clear SSSD cache:

    # systemctl stop sssd ; rm -f /var/lib/sss/db/* ; systemctl start sssd

    When there is no /etc/sssd/sssd.conf, make it as follows and should delete it later.

    # ls -l /etc/sssd/sssd.conf
# cat << 'EOF' > /etc/sssd/sssd.conf
[sssd]
enable_files_domain = true
EOF
# chmod 600 /etc/sssd/sssd.conf

  • Instead just do rm -rf /var/lib/sss/db/* Entire resolution be simplified to this.

  • Because if SSSD runs it will upgrade cache automatically. If it doesn't run there is no need to keep the cache.

Root Cause

  • useradd and usermod run "/usr/sbin/sss_cache -UG", which fails because of SYSDB_VERSION differences.
  • Default value of enable_files_domain changed to false.
  • So that not-explicitly-configured SSSD doesn't run automatically anymore.
  • But left over cache files are present on the disk.
  • sss_cache (being called from shadow-utils' user add/del) complains that version of cache db is too old.

Diagnostic Steps

# useradd testuser1
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
useradd: sss_cache exited with status 70
useradd: Failed to flush the sssd cache.
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
useradd: sss_cache exited with status 70
useradd: Failed to flush the sssd cache.

# id testuser1
uid=1003(testuser1) gid=1003(testuser1) groups=1003(testuser1)

# usermod --comment test testuser1
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
usermod: sss_cache exited with status 70
usermod: Failed to flush the sssd cache.
[sss_cache] [sysdb_domain_cache_connect] (0x0010): DB version too old [0.22], expected [0.23] for domain implicit_files!
Higher version of database is expected!
In order to upgrade the database, you must run SSSD.
Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.
Could not open available domains
usermod: sss_cache exited with status 70
usermod: Failed to flush the sssd cache.

# getent passwd testuser1
testuser1:x:1003:1003:test:/home/testuser1:/bin/bash

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments