3.11. Using sched_nr_migrate to Limit SCHED_OTHER Task Migration.

If a SCHED_OTHER task spawns a large number of other tasks, they will all run on the same CPU. The migration task or softirq will try to balance these tasks so they can run on idle CPUs. The sched_nr_migrate option can be set to specify the number of tasks that will move at a time. Because real-time tasks have a different way to migrate, they are not directly affected by this, however when softirq moves the tasks it locks the run queue spinlock that is needed to disable interrupts. If there are a large number of tasks that need to be moved, it will occur while interrupts are disabled, so no timer events or wakeups will happen simultaneously. This can cause severe latencies for real-time tasks when the sched_nr_migrate is set to a large value.

Procedure 3.4. Adjusting the Value of the sched_nr_migrate Variable

  1. Increasing the sched_nr_migrate variable gives high performance from SCHED_OTHER threads that spawn lots of tasks, at the expense of real-time latencies. For low real-time task latency at the expense of SCHED_OTHER task performance, the value must be lowered. The default value is 8.
  2. To adjust the value of the sched_nr_migrate variable, you can echo the value directly to /proc/sys/kernel/sched_nr_migrate: 
    ~]# echo 2 > /proc/sys/kernel/sched_nr_migrate