8.10. Initializing Consumers

Once a replication agreement is created, the consumer must be initialized; that is, the data must be physically copied from the supplier server to the consumer servers.

NOTE

Replication will not begin until the consumer is initialized.

NOTE

For large databases, the nsslapd-idletimeout setting must be set to a large enough time period (or even an unlimited period) to allow the entire database to be initialized before the operation times out. Alternatively, the nsIdleTimeoutt setting for the supplier bind DN entry can be set high enough to allow the online initialization operation to complete, without having to change the global setting.

8.10.1. When to Initialize a Consumer

Consumer initialization involves copying data from the supplier server to the consumer server. Once the subtree has been physically placed on the consumer, the supplier server can begin replaying update operations to the consumer server.
Under normal operations, the consumer should not ever have to be reinitialized. However, any time there is a chance that there is a big discrepancy between the supplier's data and the consumer's, reinitialize the consumer. For example, if the data on the supplier server is restored from backup, then all consumers supplied by that server should be reinitialize. As another example, if the supplier has not been able to contact the consumer for a long time, like a week, the supplier may determine that the consumer is too far out of date to be updated, and must be reinitialized.
The consumer can either be initialized online using the Console or manually using the command line. Online consumer initialization using the Console is an effective method of initializing a small number of consumers. However, since each replica is initialized in sequence, this method is not suited to initializing a large number of replicas. Online consumer initialization is the method to use when the consumer is initialized as part of configuring the replication agreement on the supplier server.
Manual consumer initialization using the command line is a more effective method of initializing a large number of consumers from a single LDIF file.

8.10.2. Online Consumer Initialization Using the Console

Online consumer initialization using the Console is the easiest way to initialize or reinitialize a consumer. However, for replicating across a slow link, this process can be very time-consuming, and manual consumer initialization using the command line may be a more efficient approach. This is described in more detail Section 8.10.4, “Manual Consumer Initialization Using the Command Line”.

NOTE

When a consumer server is being initialized using the online consumer creation method, all operations (including searches) on the replica are referred to the supplier server until the initialization process is completed.
To initialize or reinitialize a consumer online:
  1. Create a replication agreement.
  2. On the supplier server, on the Directory Server Console, select the Configuration tab.
  3. Expand the Replication folder, then expand the replicated database. Right-click the replication agreement, and choose Initialize Consumer from the pop-up menu.
    A message opens warning that any information already stored in the replica on the consumer will be removed.
  4. Click Yes in the confirmation box.
Online consumer initialization begins immediately. To check the status of the online consumer initialization, open the Summary tab in the Status box. If online consumer initialization is in progress, the status shows that a replica is being initialized.
To update this window, right-click the replicated database icon in the navigation tree, and choose Refresh Replication Agreements. When online consumer initialization finishes, the status changes to reflect this.
For more information about monitoring replication and initialization status, see Section 8.18, “Monitoring Replication Status”.

8.10.3. Initializing Consumers Online Using the Command Line

Online consumer initialization can be performed through the command line by adding the nsds5BeginReplicaRefresh attribute to the replication agreement entry. This attribute is absent by default, and it will be automatically deleted once the consumer initialization is complete.
  1. Find the DN of the replication agreement on the supplier server that is for the consumer to be initialized. For example:
    ldapsearch -h supplier1.example.com -p 389 -D "cn=directory manager" -w secret -s sub 
         -b cn=config "(objectclass=nsds5ReplicationAgreement)"
    This command returns all of the replication agreements configured on the supplier in LDIF format. Get the DN of the replication agreement with the consumer to be initialized. This is the replication agreement which will be edited.
  2. Edit the replication agreement, and add the nsds5BeginReplicaRefresh attribute:
    /usr/lib/mozldap/ldapmodify -D "cn=directory manager" -w secret -p 389 -h supplier1.example.com 
    
    dn: cn=ExampleAgreement,cn=replica,cn="dc=example,dc=com",cn=mapping tree,cn=config
    changetype: modify
    replace: nsds5BeginReplicaRefresh
    nsds5BeginReplicaRefresh: start
    ldapmodify does not prompt for input; simply type in the LDIF statement, and then hit enter twice when the LDIF statement is complete. Close the ldapmodify utility by hitting Ctrl+C.
To check the initialization status, do an ldapsearch for the replication agreement entry.
ldapsearch -h hostname -p port -D "cn=directory manager" -w secret -s base 
     -b 'cn=ExampleAgreement,cn="dc=example,dc=com", cn=mapping tree, cn=config' '(objectclass=*)'
If the nsds5BeginReplicaRefresh attribute is present, the initialization is still in progress. If the initialization is complete, then the attribute nsds5ReplicaLastInitStatus shows the status. If the initialization was successful, the value of nsds5ReplicaLastInitStatus is Total update succeeded. If the initialization was not successful, this attribute shows information about the error; check the error logs for both the supplier and consumer for additional information.
The replication monitoring attributes are described in more detail in the Directory Server Configuration and Command Reference.

8.10.4. Manual Consumer Initialization Using the Command Line

Manual consumer initialization using the command line is the fastest method of consumer initialization for sites that are replicating very large numbers of entries. However, the manual consumer initialization process is more complex than the online consumer initialization process. Red Hat suggests using the manual process whenever the online process is inappropriate due to performance concerns.
Initializing or reinitializing a server manually has three steps:
  1. Create a replication agreement.
  2. Export the replica on the supplier server to an LDIF file.
  3. Import the LDIF file with the supplier replica contents to the consumer server.

8.10.4.1. Exporting a Replica to LDIF

There are three ways to convert a replica database to LDIF:
  • When creating a replication agreement, by selecting Create consumer initialization file in the Initialize Consumer dialog box of the Replication Agreement Wizard.
  • From the Directory Server Console, by right-clicking the replication agreement under the Replication folder and choosing Create LDIF File from the pop-up menu.
  • From the command line by using the export command, as described in Section 4.2.3, “Exporting to LDIF from the Command Line”. Exporting to LDIF with any of the command-line tools requires using an option to export the database as a replica; this means that the exported LDIF contains the proper entries to initialize the consumer when the LDIF is imported.
    For the db2ldif and db2ldif.pl scripts, this is the -r option. For example:
    db2ldif -r -n database1 -a /export/output.ldif
    For the cn=export, cn=tasks, cn=config entry, this is the nsExportReplica attribute.
     /usr/lib/mozldap/ldapmodify -a -D "cn=directory manager" -w secret -p 389 -h server.example.com
    
     dn: cn=example export, cn=export, cn=tasks, cn=config
     objectclass: extensibleObject
     cn: example export
     nsInstance: userRoot
     nsFilename: /home/files/example.ldif
     nsExportReplica: true

8.10.4.2. Importing the LDIF File to the Consumer Server

Import the LDIF file which contains the supplier replica contents to the consumer server by using the import features in the Directory Server Console or by using either the ldif2db script or ldif2db.pl script. Both import methods are described in Section 4.1.4, “Importing from the Command Line”.

NOTE

With the ldif2db.pl script, the LDIF file import operation does not require a server restart. For more information on command-line scripts, see the Directory Server Configuration and Command Reference.

8.10.5. Filesystem Replica Initialization

A very large database, such as one with several million entries, can take an hour or more to initialize a consumer from the Console or even with manual initialization. To save time, use filesystem replica initialization.
Directory Server has the capability to initialize a replica using the database files from the supplier server. This avoids the need to rebuild the consumer database and can be done at essentially the speed of the network between the two servers by transferring the files with FTP or NFS, for example. Instead of sending entries via LDAP to replica servers, filesystem replica initialization populates the new database on the destination server by backing up the supplier database on one server and restoring the database on the destination server.
This method of initializing consumers is especially useful in replication over wide-area networks or over networks with slow or unstable connections.
For smaller databases, Red Hat recommends using manual initialization or initialize consumers from the Console.

NOTE

The destination server must have the same architecture and the same bit size as the supplier server for the initialization to succeed. For example, Red Hat Enterprise Linux 5 (32-bit) to Red Hat Enterprise Linux 5 (32-bit).

8.10.5.1. Initializing the Consumer Replica from the Backup Files

  1. Create a new database on the destination server to match the database from the source server.
    Before initializing the consumer from the backup files, be certain that the appropriate database has been created on the destination server so that the database exists to be restored and initialized.
  2. Enable replication on the backend as a dedicated consumer.
  3. If there is already a replication agreement to that host and port, then replication should resume immediately after running the restore script. Otherwise, create the replication agreement on the source server (or whatever server is the supplier), and select the Do not initialize consumers at this time radio button.
  4. Stop the source Directory Server if it is running. For example:
    service dirsrv stop slapd-example
  5. From the command line, run the db2bak utility, and archive the current directory installation.
    /etc/dirsrv/slapd-instance_name/db2bak
    In addition, a new backup can be created by hitting the Back Up Directory Server button in the Tasks tab of the Directory Server Console and then putting the name of the archive directory in the Directory:... field. Alternatively, select any previous back-up to initialize the consumer.
    This backup information is recovered in the destination replica.
  6. Restart the source Directory Server. For example:
    service dirsrv start slapd-example
  7. Copy the archived files to a directory on the destination server, such as archiveDirectory.
  8. Stop the destination Directory Server if it is running.
    service dirsrv stop slapd-example2
  9. On the destination server, restore the archives with the bak2db script, using the optional -n parameter to specify the backend instance name. This -n parameter is similar to the -n used with ldif2db and db2ldif. For example:
    /usr/lib/dirsrv/slapd-example2/bak2db /tmp/archiveDirectory -n userRoot
  10. Restart the destination Directory Server. For example:
    service dirsrv start slapd-example2
Replication will begin on schedule as soon as the destination server is restarted.
For more information on using these scripts, see the Directory Server Configuration and Command Reference.