Chapter 19. Troubleshooting IdM replica installation

The following sections describe the process for gathering information about a failing IdM replica installation, and how to resolve some common installation issues.

19.1. Reviewing IdM replica installation errors

When you install an Identity Management (IdM) replica, debugging information is appended to the following log files on the replica:

  • /var/log/ipareplica-install.log
  • /var/log/ipareplica-conncheck.log
  • /var/log/ipaclient-install.log
  • /var/log/httpd/error_log
  • /var/log/dirsrv/slapd-INSTANCE-NAME/access
  • /var/log/dirsrv/slapd-INSTANCE-NAME/errors
  • /var/log/ipaserver-install.log

The replica installation process also appends debugging information to the following log files on the IdM server the replica is contacting:

  • /var/log/httpd/error_log
  • /var/log/dirsrv/slapd-INSTANCE-NAME/access
  • /var/log/dirsrv/slapd-INSTANCE-NAME/errors

The last line of each log file reports success or failure, and ERROR and DEBUG entries provide additional context.

To troubleshoot a failing IdM replica installation, review the errors at the end of these log files on both hosts (replica and server) and use this information to resolve any corresponding issues.

Prerequisites

  • You must have root privileges to display the contents of IdM log files.

Procedure

  1. Use the tail command to display the latest errors from the primary log file /var/log/ipareplica-install.log. The following example displays the last 10 lines.

    [user@replica ~]$ sudo tail -n 10 /var/log/ipareplica-install.log
    [sudo] password for user:
      func(installer)
    File "/usr/lib/python3.6/site-packages/ipaserver/install/server/replicainstall.py", line 424, in decorated
      func(installer)
    File "/usr/lib/python3.6/site-packages/ipaserver/install/server/replicainstall.py", line 785, in promote_check
      ensure_enrolled(installer)
    File "/usr/lib/python3.6/site-packages/ipaserver/install/server/replicainstall.py", line 740, in ensure_enrolled
      raise ScriptError("Configuration of client side components failed!")
    
    2020-05-28T18:24:51Z DEBUG The ipa-replica-install command failed, exception: ScriptError: Configuration of client side components failed!
    2020-05-28T18:24:51Z ERROR Configuration of client side components failed!
    2020-05-28T18:24:51Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information
  2. To review the log file interactively, open the end of the log file using the less utility and use the and arrow keys to navigate.

    [user@replica ~]$ sudo less -N +G /var/log/ipareplica-install.log
  3. (Optional) While /var/log/ipareplica-install.log is the primary log file for a replica installation, you can gather additional troubleshooting information by repeating this review process with additional files on the replica and the server.

    On the replica:

    [user@replica ~]$ sudo less -N +G /var/log/ipareplica-conncheck.log
    [user@replica ~]$ sudo less -N +G /var/log/ipaclient-install.log
    [user@replica ~]$ sudo less -N +G /var/log/httpd/error_log
    [user@replica ~]$ sudo less -N +G /var/log/dirsrv/slapd-INSTANCE-NAME/access
    [user@replica ~]$ sudo less -N +G /var/log/dirsrv/slapd-INSTANCE-NAME/errors
    [user@replica ~]$ sudo less -N +G /var/log/ipaserver-install.log

    On the server:

    [user@server ~]$ sudo less -N +G /var/log/httpd/error_log
    [user@server ~]$ sudo less -N +G /var/log/dirsrv/slapd-INSTANCE-NAME/access
    [user@server ~]$ sudo less -N +G /var/log/dirsrv/slapd-INSTANCE-NAME/errors

Additional resources

  • If you are unable to resolve a failing replica installation, and you have a Red Hat Technical Support subscription, open a Technical Support case at the Red Hat Customer Portal and provide an sosreport of the replica and an sosreport of the server.
  • The sosreport utility collects configuration details, logs and system information from a RHEL system. For more information on the sosreport utility, see What is an sosreport and how to create one in Red Hat Enterprise Linux?.

19.2. Reviewing IdM CA installation errors

Installing the Certificate Authority (CA) service on an Identity Management (IdM) replica appends debugging information to several locations on the replica and the IdM server the replica communicates with.

Table 19.1. On the replica (in order of recommended priority):

LocationDescription

/var/log/pki/pki-ca-spawn.$TIME_OF_INSTALLATION.log

High-level issues and Python traces for the pkispawn installation process

journalctl -u pki-tomcatd@pki-tomcat output

Errors from the pki-tomcatd@pki-tomcat service

/var/log/pki/pki-tomcat/ca/debug.$DATE.log

Large JAVA stacktraces of activity in the core of the Public Key Infrastructure (PKI) product

/var/log/pki/pki-tomcat/ca/signedAudit/ca_audit

Audit log of the PKI product

  • /var/log/pki/pki-tomcat/ca/system
  • /var/log/pki/pki-tomcat/ca/transactions
  • /var/log/pki/pki-tomcat/catalina.$DATE.log

Low-level debug data of certificate operations for service principals, hosts, and other entities that use certificates

On the server contacted by the replica:

  • /var/log/httpd/error_log log file

Installing the CA service on an existing IdM replica also writes debugging information to the following log file:

  • /var/log/ipareplica-ca-install.log log file
Note

If a full IdM replica installation fails while installing the optional CA component,no details about the CA are logged; a message is logged in the /var/log/ipareplica-install.log file indicating that the overall installation process failed. Red Hat recommends reviewing the log files listed above for details specific to the CA installation failure.

The only exception to this behavior is when you are installing the CA service and the root CA is an external CA. If there is an issue with the certificate from the external CA, errors are logged in /var/log/ipareplica-install.log.

To troubleshoot a failing IdM CA installation, review the errors at the end of these log files and use this information to resolve any corresponding issues.

Prerequisites

  • You must have root privileges to display the contents of IdM log files.

Procedure

  1. To review a log file interactively, open the end of the log file using the less utility and use the and arrow keys to navigate, while searching for ScriptError entries. The following example opens /var/log/pki/pki-ca-spawn.$TIME_OF_INSTALLATION.log.

    [user@server ~]$ sudo less -N +G /var/log/pki/pki-ca-spawn.20200527185902.log
  2. Gather additional troubleshooting information by repeating this review process with all the log files listed above.

Additional resources

19.3. Removing a partial IdM replica installation

If an IdM replica installation fails, some configuration files may be left behind. Additional attempts to install the IdM replica can fail and the installation script reports that IPA is already configured.

[root@server ~]# ipa-replica-install
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

IPA server is already configured on this system.
If you want to reinstall the IPA server, please uninstall it first using 'ipa-server-install --uninstall'.
The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

To resolve this issue, uninstall IdM software from the replica, remove the replica from the IdM topology, and retry the installation process.

Prerequisites

  • You must have root privileges.

Procedure

  1. Uninstall the IdM server software on the host you are trying to configure as an IdM replica.

    [root@replica ~]# ipa-server-install --uninstall
  2. On all other servers in the topology, use the ipa server-del command to delete any references to the replica that did not install properly.

    [root@other-replica ~]# ipa server-del replica.idm.example.com
  3. Attempt installing the replica.
  4. If you continue to experience difficulty installing an IdM replica because of repeated failed installations, reinstall the operating system.

    One of the requirements for installing an IdM replica is a clean system without any customization. Failed installations may have compromised the integrity of the host by unexpectedly modifying system files.

Additional resources

19.4. Resolving invalid credential errors

If an IdM replica installation fails with an Invalid credentials error, the system clocks on the hosts may be out of sync with each other:

[27/40]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 15 seconds elapsed
[ldap://server.example.com:389] reports: Update failed! Status: [49 - LDAP error: Invalid credentials]

[error] RuntimeError: Failed to start replication
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

ipa.ipapython.install.cli.install_tool(CompatServerReplicaInstall): ERROR    Failed to start replication
ipa.ipapython.install.cli.install_tool(CompatServerReplicaInstall): ERROR    The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

If you use the --no-ntp or -N options to attempt the replica installation while clocks are out of sync, the installation fails because services are unable to authenticate with Kerberos.

To resolve this issue, synchronize the clocks on both hosts and retry the installation process.

Prerequisites

  • You must have root privileges to change system time.

Procedure

  1. Synchronize the system clocks manually or with chronyd (ntp is no longer supported in RHEL 8).

    • Synchronizing manually:

      Display the system time on the server and set the replica’s time to match.

      [user@server ~]$ date
      Thu May 28 21:03:57 EDT 2020
      
      [user@replica ~]$ sudo timedatectl set-time '2020-05-28 21:04:00'
    • Synchronizing with chronyd:

      Refer to Using the Chrony suite to configure NTP to configure and set system time with chrony tools.

  2. Attempt the IdM replica installation again.

Additional resources

  • If you are unable to resolve a failing replica installation, and you have a Red Hat Technical Support subscription, open a Technical Support case at the Red Hat Customer Portal and provide an sosreport of the replica and an sosreport of the server.
  • The sosreport utility collects configuration details, logs and system information from a RHEL system. For more information on the sosreport utility, see What is an sosreport and how to create one in Red Hat Enterprise Linux?.

19.5. Additional resources