4.3. Interrupts and IRQ Tuning

An interrupt request (IRQ) is a request for service, sent at the hardware level. Interrupts can be sent by either a dedicated hardware line, or across a hardware bus as an information packet (a Message Signaled Interrupt, or MSI).

When interrupts are enabled, receipt of an IRQ prompts a switch to interrupt context. Kernel interrupt dispatch code retrieves the IRQ number and its associated list of registered Interrupt Service Routines (ISRs), and calls each ISR in turn. The ISR acknowledges the interrupt and ignores redundant interrupts from the same IRQ, then queues a deferred handler to finish processing the interrupt and stop the ISR from ignoring future interrupts.

The /proc/interrupts file lists the number of interrupts per CPU per I/O device. It displays the IRQ number, the number of that interrupt handled by each CPU core, the interrupt type, and a comma-delimited list of drivers that are registered to receive that interrupt. (Refer to the proc(5) man page for further details: man 5 proc)

IRQs have an associated "affinity" property, smp_affinity, which defines the CPU cores that are allowed to execute the ISR for that IRQ. This property can be used to improve application performance by assigning both interrupt affinity and the application's thread affinity to one or more specific CPU cores. This allows cache line sharing between the specified interrupt and application threads.

The interrupt affinity value for a particular IRQ number is stored in the associated /proc/irq/IRQ_NUMBER/smp_affinity file, which can be viewed and modified by the root user. The value stored in this file is a hexadecimal bit-mask representing all CPU cores in the system.

As an example, to set the interrupt affinity for the Ethernet driver on a server with four CPU cores, first determine the IRQ number associated with the Ethernet driver:

# grep eth0 /proc/interrupts
32:   0     140      45       850264      PCI-MSI-edge      eth0

Use the IRQ number to locate the appropriate smp_affinity file:

# cat /proc/irq/32/smp_affinity 
f

The default value for smp_affinity is f, meaning that the IRQ can be serviced on any of the CPUs in the system. Setting this value to 1, as follows, means that only CPU 0 can service this interrupt:

# echo 1 >/proc/irq/32/smp_affinity
# cat /proc/irq/32/smp_affinity
1

Commas can be used to delimit smp_affinity values for discrete 32-bit groups. This is required on systems with more than 32 cores. For example, the following example shows that IRQ 40 is serviced on all cores of a 64-core system:

# cat /proc/irq/40/smp_affinity
ffffffff,ffffffff

To service IRQ 40 on only the upper 32-cores of a 64-core system, you would do the following:

# echo 0xffffffff,00000000 > /proc/irq/40/smp_affinity
# cat /proc/irq/40/smp_affinity
ffffffff,00000000