9.4. ds-logpipe.py

The named pipe log script can replace any of the Directory Server log files (access, errors, and audit) with a named pipe. That pipe can be attached to another script which can process the log data before sending it to output, such as only writing lines that match a certain pattern or are of a certain event type.
Using a named pipe script provides flexibility:
  • The error log level can be set very high for diagnosing an issue to create a log of only the last few hundred or thousand log messages, without a performance hit.
  • Messages can be filtered to keep only certain events of interest. For example, the named pipe script can record only failed BIND attempts in the access log, and other events are discarded.
  • The script can be used to send notifications when events happen, like adding or deleting a user entry or when a specific error occurs.

ds-logpipe.py /path/to/named_pipe [ --user pipe_user ] [ --maxlines number ] [[ --serverpidfile file.pid ] | [ --serverpid PID ]] [ --servertimeout seconds ] [ --plugin=/path/to/plugin.py | [ pluginfile.arg=value ]]


Several of the options that can be used with ds-logpipe.py have abbreviated arguments.

Table 9.6. ds-logpipe.py Options

Option Abbreviation Description
/path/to/named_pipe Required. The fully path and name of the pipe to which the server will send the logging data. If SELinux is in enforcing mode, then the named pipe must be in the instance's default log directory (/var/log/dirsrv/slapd-instance) so that the Directory Server can access and run the pipe file without violating SELinux rules.
--user -u The user ID to which the named pipe will be chowned. Any files created by plug-ins will also be owned by that user.
--maxlines -m The number of lines to keep in the buffer. The default is 1000.
--serverpidfile -s The name of the file which contains the PID of the server. By default, this is /var/run/dirsrv/slapd-instance.pid. This option allows you to start and stop the named pipe with the server process.
--serverpid The process ID for the server. The server must already be running to use this argument.
--servertimeout -t The amount of time, in seconds, to wait for the PID file to be created and for the process to be running. The default is 60 (seconds).
--plugin Gives the name of a plug-in to call which defines a function to call with each line read from the pipe. An optional pre-function can be given to call when the plug-in is loaded, and an optional post-function can be given to run when the script exits. This file must be a Python script and must end in .py. Arguments can be passed to the plug-in using the pluginfile.arg option.
pluginfile.arg Defines a plug-in argument. pluginfile is the name of the plug-in and each arg is the name of the argument for that plug-in. For example, to pass an argument name ldifinput to a plug-in named exampleplug, the argument would be exampleplug.ldifinput.

The procedures for configuring the server for named pipe logging are covered in Section 7.5, “Replacing Log Files with a Named Pipe”.

The most basic usage of the named pipe log script points to only the named pipe.

Example 9.12. Basic Named Pipe Log Script

# ds-logpipe.py /var/log/dirsrc/slapd-example/errors.pipe


When the script exits (either because it completes or because it is terminated through a SIGTERM or Ctrl+C), the script dumps the last 1000 lines of the error log to standard output.
The script can be run in the background, and you can interactively monitor the output. In that case, the command kill -1 %1 can be used to tell the script to dump the last 1000 lines of the buffer to stdout, and continue running in the background.

Example 9.13. Running the Named Pipe Log Script in the Background

# ds-logpipe.py /var/log/dirsrc/slapd-example/errors.pipe &
To simply dump the last 1000 lines when the script exits (or is killed or interrupted) and save the output to a file automatically, redirect the script output to a user-defined file.

Example 9.14. Saving the Output from the Named Pipe Log Script

# ds-logpipe.py /var/log/dirsrc/slapd-example/errors.pipe > /etc/dirsrv/myerrors.log 2>&1
The named pipe script can be configured to start and stop automatically with the Directory Server process. This requires the name of the server's PID file to which to write the script's PID when the script is running, with the -s argument. The PID for the server can be reference either by pointing to the server PID file or by giving the actual process ID number (if the server process is already running).

Example 9.15. Specifying the Serve PID

# ds-logpipe.py /var/log/dirsrc/slapd-example/errors.pipe --serverpidfile /var/run/dirsrv/slapd-example.pid
A plug-in can be called to read the log data from the named pipe and perform some operation on it.

Example 9.16. Named Pipe Log Script with a Related Plug-in

# ds-logpipe.py /var/log/dirsrc/slapd-example/errors.pipe --plugin=/usr/share/dirsrv/data/logregex.py logregex.regex="warning"
In Example 9.16, “Named Pipe Log Script with a Related Plug-in”, only log lines containing the string warning are stored in the internal buffer and printed when the script exits.
If no plug-in is passed with the script arguments, the script just buffers 1000 log lines (by default) and prints them upon exit. There are two plug-ins provided with the script:
  • logregex.py keeps only log lines that match the given regular expression. The plug-in argument has the format logregex.regex=pattern to specify the string or regular expression to use. There can be multiple logregex.regex arguments which are all treated as AND statements. The error log line must match all given arguments. To allow any matching log lines to be records (OR), use a single logregex.regex argument with a pipe (|) between the strings or expressions. See the pcre or Python regular expression documentation for more information about regular expressions and their syntax.
  • failedbinds.py logs only failed BIND attempts, so this plug-in is only used for the access log. This takes the option failedbinds.logfile=/path/to/access.log, which is the file that the actual log messages are written to. This plug-in is an example of a complex plug-in that does quite a bit of processing and is a good place to reference to do other types of access log processing.