3.10. Using sched_nr_migrate to limit SCHED_OTHER task migration.

If a SCHED_OTHER task spawns a large number of other tasks, they will all run on the same CPU. The migration task or softirq will try to balance these tasks so they can run on idle CPUs. The sched_nr_migrate option can be set to specify the number of tasks that will move at a time. Because realtime tasks have a different way to migrate, they are not directly affected by this, however when softirq moves the tasks it locks the run queue spinlock that is needed to disable interrupts. If there are a large number of tasks that need to be moved, it will occur while interrupts are disabled, so no timer events or wakeups will happen simultaneously. This can cause severe latencies for realtime tasks when the sched_nr_migrate is set to a large value.

Procedure 3.4. Adjusting the value of the sched_nr_migrate variable

  1. Increasing the sched_nr_migrate variable gives high performance from SCHED_OTHER threads that spawn lots of tasks, at the expense of realtime latencies. For low realtime task latency at the expense of SCHED_OTHER task performance, the value must be lowered. The default value is 8.
  2. To adjust the value of the sched_nr_migrate variable, you can echo the value directly to /proc/sys/kernel/sched_nr_migrate:
    # echo 2 > /proc/sys/kernel/sched_nr_migrate