3. Optimizing System Performance

There are a limited number of areas where settings on the machine on which the Directory Server instance is running can be changed to improve Directory Server performance. Basically, the machine can be configured to grant the Directory Server more system resources (TCP ports and file descriptors) which helps it handle higher loads with better performance. The host machine can also be configured to allow better client access to the server (firewalls and DNS), which doesn't have a direct impact on Directory Server performance, but does affect using the server.

TIP

The Red Hat Enterprise Linux documentation includes a whitepaper on tuning the Red Hat Enterprise Linux system settings to improve Oracle database performance and is available at http://www.redhat.com/docs/manuals/enterprise/RHELTuningandOptimizationforOracleV11.pdf. Similar adjustments can be made to any Red Hat Enterprise Linux system hosting a Red Hat Directory Server instance, and that performance tuning whitepaper is a good resource to help with Directory Server system tuning.

3.1. Tuning and Sizing System Memory

There are several different areas of the Red Hat Enterprise Linux machine that can be modified to improve the performance of the Directory Server itself.

NOTE

The cache settings here relate to the cache allowed to the Directory Server process, not to any cache settings in the Directory Server databases.

3.1.1. Setting the Minimum System Page Cache

Although Red Hat Enterprise Linux sets the page cache size dynamically, it is possible to require a minimum number of free pages by setting the vm.min_free_kbytes parameter in the system configuration. For example:
# echo vm.min_free_kbytes=1024 >> /etc/sysctl.conf

3.1.2. Managing the Swap Space

Long-running processes (and Directory Server is normally a long-running process) don't use all of the pages that they obtain. Creating a swap space to partition the processes can improve the system's performance because the system memory is used for disk cache rather than page cache.
The /proc/sys/vm/swappiness kernel parameter sets a threshold for when processes should be swapped out in favor of I/O caching. The value of this is a percentage; the higher the percentage, the higher the I/O cache and the faster that pages are swapped. The default value is 60. This can be changed by editing the system configuration as root:
# echo vm.swappiness=10 >> /etc/sysctl.conf
Constant swapping must be avoided. If swap usage is high or increasing, then increase the RAM (ideally) or the swap size.

3.1.3. Tuning for 32-Bit Systems

Ideally, Red Hat Directory Server will be run on 64-bit systems which naturally have the address space to handle very large databases and operations. Still, 32-bit systems can be optimized to the best possible performance.
First, use the hugemem kernel for databases that are 8GB or larger. This is the default kernel, anyway, in Red Hat Enterprise Linux 5.
Next, configure huge pages on the system by setting how many large memory pages should be allocated and kept in RAM for shared memory:
  1. Check the amount of memory available for huge pages.
      grep Hugepagesize /proc/meminfo
    Hugepagesize:     4096 kB
    This means that each huge page is 4GB.
  2. Set the number of huge pages that can be used through the nr_hugepages kernel parameter.
    # echo "vm.nr_hugepages=512" >> /etc/sysctl.conf
  3. Check the new huge page settings.
    The defined value for huge pages is viewable in Hugepagesize, while the total number of huge pages (set in nr_hugepages) is shown in HugePages_Total, both in the /proc/meminfo:
    HugePages_Total:     512
    HugePages_Free:      512
    HugePages_Rsvd:      0
    Hugepagesize:     4096 kB
  4. For the Directory Server databases to be able to use huge pages, add or set the memlock parameter for the Directory Server user (nobody by default) to a high enough seting to allow them to lock that much memory.
    vim /etc/security/limits.conf
    
    nobody        soft      memlock           1048576
    nobody        hard      memlock           1048576

3.2. Using dsktune for Preliminary Tuning

When a Directory Server instance is first set up, the setup command automatically runs a tool called dsktune which checks the current machine settings and compares them to the requirements for Directory Server.
This tool can be run again at any point to check whether any system settings should be adjusted.
[jsmith@example ~]$ dsktune
Red Hat Directory Server system tuning analysis version 10-AUGUST-2007.

NOTICE : System is i686-unknown-linux2.6.18-164.2.1.el5 (2 processors).


NOTICE : The net.ipv4.tcp_keepalive_time is set to 7200000 milliseconds
(120 minutes).  This may cause temporary server congestion from lost
client connections.

WARNING: There are only 1024 file descriptors (hard limit) available, which
limit the number of simultaneous connections.  

WARNING: There are only 1024 file descriptors (soft limit) available, which
limit the number of simultaneous connections.

3.3. Tuning File Descriptors

The Directory Server is multi-threaded and efficiently handles many simultaneous requests and large numbers of connections.
Most of the time, the only limit on Directory Server threads is whatever limit the operating system imposes on the number of file descriptors available in the whole system and per process. The Directory Server uses file descriptors in a range of directory operations — logging, chaining, replication, client connections, and indexes. Adjusting the number of file descriptors for the Directory Server, then, can improve how well it handles large loads for a variety of operations.

3.3.1. Resetting the Host Machine's File Descriptors

  1. First, check the current system limit for file descriptors:
    cat /proc/sys/fs/file-max
  2. If the setting is lower than 64000, edit the /etc/sysctl.conf file, and reset the fs.file-max parameter:
    fs.file-max = 64000
  3. Then increase the maximum number of open files on the system by editing the /etc/security/limits.conf configuration file. Add the following entry:
    *        -        nofile        8192
  4. Edit the /etc/pam.d/system-auth, and add this entry:
    session required /lib/security/$ISA/pam_limits.so
  5. Reboot the machine to apply the changes.
Additionally, the Directory Server keeps some file descriptors in reserve and is used by process like file I/O (e.g. log files) and outgoing network connections. The number of simultaneous connections the server can handle is set by the maximum number of server descriptors minus the assigned number of reserve descriptors. The file descriptors parameters can be reset in the Directory Server configuration to take advantage of the higher system settings.

3.3.2. Setting Directory Server File Descriptor Values

For performance, the most relevant attribute is the nsslapd-maxdescriptor attribute. This tells the Directory Server how many file descriptors it has available. (This value cannot be larger than the system setting, or the server ignores it.)
Reserve file descriptors rarely need to be reset. Decreasing the number of reserve file descriptors can increase the number of LDAP clients that can connect to the server. However, too few reserve file descriptors can still hirt directory performance by causing server errors. There are three instances that can require raising the numbr of reserve file descriptors:
  • The server is replicating to a large number of consumer servers (more than 10), and/or the server is maintaining a large number of index files (more than 30).
  • The server is servicing a large number of LDAP connections.
  • There are error messages reporting that the server is unable to open file descriptors (the actual error message differs depending on the operation that the server is attempting to perform), but these error messages are not related to managing client LDAP connections.
To calculate the number of reserved file descriptors to set, use the following formula:
nsslapd-reservedescriptor = 20 + (NldbmBackends * 4) + NglobalIndex +
ReplicationDescriptor + ChainingBackendDescriptors + PTADescriptors + SSLDescriptors
  • NldbmBackends is the number of ldbm databases.
  • NglobalIndex is the total number of configured indexes for all databases including system indexes. (By default 8 system indexes and 17 additional indexes per database).
  • ReplicationDescriptor is eight (8) plus the number of replicas in the server that can act as a supplier or hub (NSupplierReplica).
  • ChainingBackendDescriptors is NchainingBackend times the nsOperationConnectionsLimit (a chaining or database link configuration attribute; 10 by default).
  • PTADescriptors is 3 if PTA is configured and 0 if PTA is not configured.
  • SSLDescriptors is 5 (4 files + 1 listensocket) if SSL is configured and 0 if SSL is not configured.
To reset the file descriptors for Directory Server:
  1. Edit the /etc/sysconfig/dirsrv file and add a new line setting the process ulimit.
    ulimit -n 8192
  2. Update the nsslapd-maxdescriptors and nsslapd-reservedescriptors values.
    /usr/lib64/mozldap/ldapmodify -D "cn=directory manager" -w secret -p 389 -h server.example.com
    
    dn: cn=config
    changetype: modify
    replace: nsslapd-maxdescriptors
    nsslapd-maxdescriptors: 8192
    -
    replace: nsslapd-reservedescriptors
    nsslapd-reservedescriptors: 40
    -
  3. Restart the Directory Server.
    service dirsrv restart

3.4. Setting Shell Limits for the Directory Server User

The Directory Server process runs as a regular user on the host system. (By default, that user is nobody.) Once system settings have been changed, then the limits for that user can be updated.

TIP

The Red Hat Enterprise Linux documentation includes a whitepaper on tuning the Red Hat Enterprise Linux system settings to improve Oracle database performance and is available at http://www.redhat.com/docs/manuals/enterprise/RHELTuningandOptimizationforOracleV11.pdf. Similar adjustments can be made to any Red Hat Enterprise Linux system hosting a Red Hat Directory Server instance, and that performance tuning whitepaper is a good resource to help with Directory Server system tuning.

3.4.1. Setting Shell Limits for File Descriptors

  1. As root, open the system's /etc/security/limits.conf file.
  2. Add two lines that set the hard and soft limits for the file descriptors (nofile) for the Directory Server user. The soft limit sets how many file descriptors the user has available by default; the user can manually adjust that setting until they hit the hard limit.
    nobody               soft        nofile          4096
    nobody               hard        nofile          63536

    IMPORTANT

    Do not set the hard limit for the Directory Server user equal to (or higher than) the maximum number of file descriptors assigned to the system itself in /proc/sys/fs/file-max. If the hard limit is too high and the user users all of those file descriptors, then the entire system runs out of file descriptors.

3.4.2. Setting Shell Limits for the Max Number of Processes

  1. As root, open the system's /etc/security/limits.conf file.
  2. Add two lines that set the hard and soft limits for the number of processes (nproc) for the Directory Server user. The soft limit sets how many processes the user has available by default; the user can manually adjust that setting until they hit the hard limit.
    nobody      soft      nproc      2047
    nobody      hard      nproc      16384

    IMPORTANT

    Do not set the hard limit for the Directory Server user equal to (or higher than) the maximum number of file descriptors assigned to the system itself in /proc/sys/fs/file-max. If the hard limit is too high and the user users all of those file descriptors, then the entire system runs out of file descriptors.

3.5. Tuning TCP Ports

If there is high traffic on a server — like the same machine is running both a web server and a Directory Server or there are a lot of clients who access the Directory Server directly — then it may be necessary to broaden the range of TCP ports available on the machine.
echo "1024 65000" > /proc/sys/net/ipv4/ip_local_port_range

3.6. Changing Firewall Settings for the Directory Server Console

When remote clients are access the Directory Server instance, particularly the Directory Server Console or applications like Red Hat Certificate System, then make sure that the iptables configuration allows those connections past the firewall.
For example, if the instance is using the default ports and TLS/SSL is configured, then there are three ports that need to be opened in iptables.
  1. Open the iptables file.
    # vi /etc/sysconfig/iptables
  2. Add a line to the configuration for each port that is in use for the Directory Server:
    -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 389 -j ACCEPT
    -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 636 -j ACCEPT
    -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 9830 -j ACCEPT

    NOTE

    Only add the TLS/SSL port (636) if TLS/SSL is actually enabled for the server.
  3. Restart the firewall:
    # service iptables restart

3.7. DNS Requirements

It is very important that DNS and reverse DNS be working correctly on the host machine, especially if you are using TLS/SSL or Kerberos with Directory Server.
Configure the DNS resolver and the NIS domain name by the modifying the /etc/resolv.conf, /etc/nsswitch.conf, and /etc/netconfig files, and set the DNS resolver for name resolution.
Edit the /etc/defaultdomain file to include the NIS domain name. This ensures that the fully-qualified host and domain names used for the Directory Server resolve to a valid IP address and that that IP address resolves back to the correct hostname.
Reboot the machine to apply these changes.