Query/Bind speed degraded after updating to Red Hat Enterprise Linux 7 389-ds

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 7
  • Red Hat Directory Server 10
  • Red Hat Identity Management

Issue

  • After updating to Red Hat Directory Server 10, LDAP binds and initial connections are extremely slow.
  • After upgrading Red Hat Identity Management to Red Hat Enterprise Linux 7, LDAP binds and initial connections are extremely slow.

Resolution

  • Turn off the new Turbo Mode feature of 389-ds
    (you will need to type this line by line)

    # ldapmodify -D "cn=directory manager" -W
dn: cn=config
changetype: modify
replace: nsslapd-enable-turbo-mode
nsslapd-enable-turbo-mode: off

When you are fininished, press CTRL+D

Root Cause

The Turbo Mode feature in Red Hat Enterprise Linux 7 389 Directory Server (RHDS/IPA) causes worker threads to handle LDAP jobs differently.

Diagnostic Steps

To profile the activity of ns-slapd and ldapsearch to understand what precisely they are doing, strace and pstack are use to show their flow of function calls and execution.

  1. Capture pstack of ns-slapd and strace for further analysis as shown below:

    # pstack $(pidof ns-slapd) > /tmp/pstack_$(date +%d-%m-%Y_%H%M%S).txt

  2. Run strace on ns-slapd pid:

    # strace -Tttfxvoy /tmp/dirsrv.strace  -s 8192 -p $(pidof ns-slapd)

  3. From another console, run the ldapsearch command with strace.

    # strace -Tttfxvoy /tmp/ldapsearch.strace  -s 8192 ldapsearch -x -h 127.0.0.1 -D "cn=Directory manager" -w <password> -b "ou=example,ou=com" -LLL uid=<example-username>

    During analysis we notice that strace shows "accept()" always returning immediately when called. Also accept() not being called frequently enough for incoming connection workload.

    Secondly, packet captures show ldap simple bindRequest takes more than 50s in this test instance.

  4. Check to see if turbo mode is enabled or not:

    # ldapsearch -LLL -D "cn=directory manager" -W -b cn=config "(objectClass=nsslapdConfig)" nsslapd-enable-turbo-mode

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments