Chapter 3. Realtime-Specific Tuning

Once you have completed the optimization in Chapter 2, General System Tuning you are ready to start MRG Realtime specific tuning. You must have the MRG Realtime kernel installed for these procedures.

Important

Do not attempt to use the tools in this section without first having completed Chapter 2, General System Tuning. You will not see a performance improvement.
When are you ready to begin MRG Realtime tuning, perform these steps first, as they will provide the greatest benefit:
When you have completed all the tuning suggestions in this chapter, move on to Chapter 4, Application Tuning and Deployment

3.1. Setting scheduler priorities

The MRG Realtime kernel allows fine grained control of scheduler priorities. It also allows application level programs to be scheduled at a higher priority than kernel threads. This is useful but it can also carry consequences. It is possible that it will cause the system to hang and other unpredictable behavior if crucial kernel processes are prevented from running as needed. Ultimately the correct settings are workload dependent.
Priorities are defined in groups, with some groups dedicated to certain kernel functions:

Table 3.1. Priority Map

Priority Threads Description
1 Low priority kernel threads Priority 1 is usually reserved for those tasks that need to be just above SCHED_OTHER
2 - 49 Available for use Range used for typical application priorities
50
Default hard-IRQ value
51 - 98 High priority threads
Use this range for threads that execute periodically and must have quick response times. Do not use this range for cpu-bound threads as you will starve interrupts.
99 Watchdogs and migration System threads that must run at the highest priority

Procedure 3.1. Using rtctl to Set Priorities

  1. Priorities are set using a series of levels, ranging from 0 (lowest priority) to 99 (highest priority). The system startup script rtctl initializes the default priorities of the kernel threads. By requesting the status of the rtctl service, you can view the priorities of the various kernel threads.
    # service rtctl status
    2   TS      - [kthreadd]
    3   FF     99 [migration/0]
    4   FF     99 [posix_cpu_timer]
    5   FF     50 [softirq-high/0]
    6   FF     50 [softirq-timer/0]
    7   FF     90 [softirq-net-tx/]
    ...[output truncated]...
    
    The output is in the format:
    [PID] [scheduler policy] [priority] [process name]
    
    In the scheduler policy field, a value of TS indicates a policy of normal and FF indicates a policy of FIFO (first in, first out).
  2. The rtctl system startup script relies on the /etc/rtgroups file.
    To change this file, open /etc/rtgroups in your preferred text editor.
    kthreads:*:1:*:\[.*\]
    watchdog:f:99:*:\[watchdog.*\]
    migration:f:99:*:\[migration\/.*\]
    softirq:f:70:*:\[.*(softirq|sirq).*\]
    softirq-net-tx:f:75:*:\[(softirq|sirq)-net-tx.*\]
    softirq-net-rx:f:75:*:\[(softirq|sirq)-net-rx.*\]
    softirq-sched:f:1:*:\[(softirq|sirq)-sched\/.*\]
    rpciod:f:65:*:\[rpciod.*\]
    lockd:f:65:*:\[lockd.*\]
    nfsd:f:65:*:\[nfsd.*\]
    hardirq:f:85:*:\[(irq|IRQ)[\-_/].*\]
    
  3. Each line represents a process. You can change the priority of the process by adjusting the parameters. The entries in this file are in the format:
    [group name]:[scheduler policy]:[scheduler priority]:[regular expression]
    
    In the scheduler policy field, the following values are accepted:
    o Sets a policy of other. If the policy is set to o, the scheduler priority field will be set to 0 and ignored.
    b Sets a policy of batch.
    f Sets a policy of FIFO.
    * If the policy is set to *, no change will be made to to any matched thread policy.
    The regular expression field matches the thread name to be modified.
  4. After editing the file, you will need to restart the rtctl service to reload it with the new settings:
    # service rtctl stop
    
    # service rtctl start
    Setting kernel thread priorities: done
    
Related Manual Pages
For more information, or for further reading, the following man pages are related to the information given in this section.
  • rtctl(1)
  • rtgroups(5)