3.14. Offloading RCU Callbacks

The Read-Copy-Update (RCU) system is a lockless mechanism for mutual exclusion inside the kernel. As a consequence of performing RCU operations, call-backs are sometimes queued on CPUs to be performed at a future moment when removing memory is safe.
RCU callbacks can be offloaded using the rcu_nocbs and rcu_nocb_poll kernel parameters.
  • To remove one or more CPUs from the candidates for running RCU callbacks, specify the list of CPUs in the rcu_nocbs kernel parameter, for example: 
    rcu_nocbs=1,4-6
    or 
    rcu_nocbs=3
    The second example instructs the kernel that CPU 3 is a no-callback CPU. This means that RCU callbacks will not be done in the rcuc/$CPU thread pinned to CPU 3, but in the rcuo/$CPU thread, which can be moved to a housekeeping CPU, relieving CPU 3 from doing RCU callbacks job.
    To move RCU callback threads to the housekeeping CPU, use the tuna -t rcu* -c X -m command, where X denotes the housekeeping CPU. For example, in a system where CPU 0 is the housekeeping CPU, all RCU callback threads can be moved to CPU 0 using this command: 
    ~]# tuna -t rcu* -c 0 -m
    This relieves all CPUs other than CPU 0 from doing RCU work.
  • Although the RCU offload threads can perform the RCU callbacks on another CPU, each CPU is responsible for awakening the corresponding RCU offload thread. To relieve each CPU from the responsibility of awakening their RCU offload threads, set the rcu_nocb_poll kernel parameter: 
    rcu_nocb_poll
    With rcu_nocb_poll set, the RCU offload threads will be periodically raised by a timer to check if there are callbacks to run.
A common use case for these two options is:
  1. Using rcu_nocbs=cpulist to allow the user to move all RCU offload threads to a housekeeping CPU;
  2. Setting rcu_nocb_poll to relieve each CPU from the responsibility awakening their RCU offload threads.
This combination reduces the interference on CPUs that are specialized for the user's workload.
For more information about RCU tuning on real-time, see Avoiding RCU Stalls in the real-time kernel.