satellite-installer fails with error "Failed to generate new NSS database at /etc/pki/katello/nssdb with password file /etc/pki/katello/nss_db_password-file" when FIPS mode is enabled.

Issue

• The execution of satellite-installer command fails and the following errors are logged in the /var/log/foreman-installer/satellite.log file.

  2023-06-14 14:24:15 [DEBUG ] [configure] /Stage[main]/Certs::Ssltools::Nssdb/Nssdb[/etc/pki/katello/nssdb]: Starting to evaluate the resource (477 of 2110)
  2023-06-14 14:24:15 [DEBUG ] [configure] Executing: '/bin/certutil -K -d /etc/pki/katello/nssdb -f /etc/pki/katello/nss_db_password-file'
  2023-06-14 14:24:15 [DEBUG ] [configure] Unable to verify NSS database at /etc/pki/katello/nssdb with password file /etc/pki/katello/nss_db_password-file: Execution of '/bin/certutil -K -d /etc/pki/katello/nssdb -f /etc/pki/katello
  /nss_db_password-file' returned 255: certutil: Checking token "NSS FIPS 140-2 Certificate DB" in slot "NSS FIPS 140-2 User Private Key Services"
  2023-06-14 14:24:15 [DEBUG ] [configure] certutil: could not authenticate to token NSS FIPS 140-2 Certificate DB.: SEC_ERROR_IO: An I/O error occurred during security authorization.
  2023-06-14 14:24:15 [DEBUG ] [configure] Executing: '/bin/certutil -N -d /etc/pki/katello/nssdb -f /etc/pki/katello/nss_db_password-file'
  2023-06-14 14:24:15 [ERROR ] [configure] Failed to generate new NSS database at /etc/pki/katello/nssdb with password file /etc/pki/katello/nss_db_password-file: Execution of '/bin/certutil -N -d /etc/pki/katello/nssdb -f /etc/pki/katello/nss_db_password-file' returned 255: certutil: Could not set password for the slot: SEC_ERROR_INVALID_PASSWORD: Password entered is invalid. Please pick a different one.
  2023-06-14 14:24:15 [ERROR ] [configure] Wrapped exception:
  2023-06-14 14:24:15 [ERROR ] [configure] Execution of '/bin/certutil -N -d /etc/pki/katello/nssdb -f /etc/pki/katello/nss_db_password-file' returned 255: certutil: Could not set password for the slot: SEC_ERROR_INVALID_PASSWORD: Password entered is invalid. Please pick a different one.
  2023-06-14 14:24:15 [ERROR ] [configure] /Stage[main]/Certs::Ssltools::Nssdb/Nssdb[/etc/pki/katello/nssdb]/ensure: change from 'absent' to 'present' failed: Failed to generate new NSS database at /etc/pki/katello/nssdb with password file /etc/pki/katello/nss_db_password-file: Execution of '/bin/certutil -N -d /etc/pki/katello/nssdb -f /etc/pki/katello/nss_db_password-file' returned 255: certutil: Could not set password for the slot: SEC_ERROR_INVALID_PASSWORD: ********
  2023-06-14 14:24:15 [DEBUG ] [configure] /Stage[main]/Certs::Ssltools::Nssdb/Nssdb[/etc/pki/katello/nssdb]: Evaluated in 0.13 seconds
  

• It occurs only on a FIPS enabled Red Hat Satellite\Capsule server when the katello_agent feature is enabled and further affects the startup of the qpidd service.