Change IRQ number in RHEL 6.4

Latest response

Hi Guys

We have a HP Proliant DL380p Gen8 Server running with RHEL 6.4. This server also holds 2 Digium T1 Telephone cards. These cards share the IRQ 5 with other onboard PCI cards/devices. However while running some performance test with the card, we noticed that we are not able to obtain the maximum performance from these cards as they are sharing the IRQs with other devices. If we could assign a dedicated IRQ for these 2 devices, we would be able to obtain the best performance.

On that note, we contacted the Server manufacturer and the Card manufacturer. However Card manufacturer said there is nothing we can modify from their driver configuration to assign a different IRQ. And so the server manufacturer mentioned that we can't assign a dedicated IRQ for this device and need to check with OS side if they would all to do it.

That brings me here to this discussion forum. Could some experts on this field please advise if this is something that is doable from the RHEL OS? If it is possible, what is the right procedure to get it done?

Looking to hear from the experts here in this forum...

Regards
Jo

Attachments

Responses

Hello
There is no way to change the IRQ the way you want to do it as far as I am aware of.

Howewver, refer to this document from HP for tuning for low latency.
http://h20565.www2.hp.com/portal/site/hpsc/template.BINARYPORTLET/public/kb/docDisplay/resource.process/?spf_p.tpst=kbDocDisplay_ws_BI&spf_p.rid_kbDocDisplay=docDisplayResURL&javax.portlet.begCacheTok=com.vignette.cachetoken&spf_p.rst_kbDocDisplay=wsrp-resourceState%3DdocId%253Demr_na-c01804533-7%257CdocLocale%253D&javax.portlet.endCacheTok=com.vignette.cachetoken

There is a section in the document for RHEL to improve low latency performance. There is also a section showing you how to mask software IRQ's for CPU.

The cards you have are likely PCIE, they use MSI/MSIX so the way the interrupts are managed is different to hardware interrupts of older PCI cards. The sshared interrupts are not as significant performance degradation as cxompred to hardware IRQ's not using MSI/MSIX.

If you post a lspci here I can see if there is anything else I can help with. Poist the lspci and also the deatils of the driver.

Thanks
Laurence

Hi Jo,

How did you determine that the cards are sharing IRQs?

More information about your system and the specific cards from Digium would be helpful here.

As far as direct IRQs are concerned, the older PCI specification only had 4 IRQs shared among the entire PCI bus. In some PCI systems the IRQ line they used was directly wired to the PCI slot the card was in, others allowed the BIOS to configure the IRQ of each slot.

Most modern systems use MSI (Available in PCI since 2.2 and in PCI Express). http://en.wikipedia.org/wiki/Message_Signaled_Interrupts MSI-X allows cards to specify up to 2048 interrupts.

If your system BIOS does not allow you to configure the IRQ of the slot the cards are in and the cards do not support MSI, the only thing I can offer is to try to move the cards to a different PCI slot.

-Karl

Hi Karl

We use 24port Digium FXO card. The card model is Digium Wildcard TE420B PCI Express ISDN PRI Card with Echo Cancellation (1TE420BF).

When we enter into the Server BIOS, under the IRQ numbers, we can see the IRQ numbers assigned to each of the PCI slots. That's how we knew the numbers assigned. Then we tried to modify one of PCI slot's IRQ and that change was reflected on all other equipments that was using the same IRQ number.

Any idea?

Regards
Jo

Hi Jo,

I can't find that card on Digium's website, but I found it elsewhere, and it claims to be a PCI-Express card. In this case it should be using MSI/MSIX as Lawrence mentioned.

I'd like to see the lspci -vv and cat /proc/interrupts from the host to get a better idea of what's going on.

Hi Karl

I have attached the output of the 2 commands that you requested..

Regards
Jo

16: IR-IO-APIC-fasteoi uhci_hcd:usb3, hpilo, wct4xxp
58: IR-IO-APIC-fasteoi wct4xxp

So you have one card on IRQ 16 and one on IRQ 58, and it seems they don't support MSI/MSIX. Considering the other card isn't conflicting, it might be worth moving the card that is conflicting to another slot. That could solve the problem.

With IO-APIC there's nothing the driver/OS can do to rewire the IRQ, You may want to make sure that all processes using that IRQ are running on the same CPU. And the drivers all need to play nicely together as Jamie mentioned. You can see in your /proc/interrupts that IRQ 16 moves among CPU's whereas IRQ 58 stays on a single CPU.

Hi Karl

I will try it and see how it goes. Thanks for your support

You've already got good assistance above, but I thought it was worth pointing out a device's "willingness" to share interrupts with other devices is dictated by the device driver.

Specifically, this is done by setting a flag (SA_SHIRQ or IRQF_SHARED) when the driver runs request_irq(). The flag indicates that the device can share its IRQ with other devices. This is covered in further detail in Understanding the Linux Kernel and Linux Device Drivers.

The vendor's right that they can't modify the interrupt number assigned, but they may be able to modify the fact that the device is willing to share its IRQ with other devices.

(Edit: Just to clarify, I don't want to throw your vendor under the bus and accuse them of lying when they say they can't do anything. There might be a good reason they want their device to share interrupts. This just gives you the specific detail you could ask them to quantify using.)

Hi Guys

Thanks for your valuable support and comments on the issue that was reported. I have passed this information to the team that is handling this and will get back to you once they have tried any of the options mentioned.

Regards
Jo

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.