NMI panic due to forcedeth driver mishandling MSI interrupts.
Issue
- Server with a NIC using forcedeth driver panicked with the following call traces.
eth0: too many iterations (6) in nv_nic_irq.
eth0: too many iterations (6) in nv_nic_irq.
NMI Watchdog detected LOCKUP, CPU=11, registers:
CPU 11
Modules linked in: nfs lockd nfs_acl autofs4 sunrpc cpufreq_powersave ide_dump scsi_dump diskdump zlib_deflate dm_mod joydev button battery ac ohci_hcd ehci_hcd forcedeth ext3 jbd sata_nv libata mptscsih mptsas mptspi mptscsi mptbase sd_mod scsi_mod
Pid: 0, comm: swapper Not tainted 2.6.9-67.0.22.ELlargesmp
RIP: 0010:[<ffffffff803100c3>] <ffffffff803100c3>{.text.lock.spinlock+49}
RSP: 0018:0000010419647bc8 EFLAGS: 00000082
RAX: 0000000000000000 RBX: 0000010418e2e380 RCX: 0000000093808538
RDX: 0000010417abd280 RSI: 0000010093808538 RDI: 0000010418e2e380
RBP: 0000010418e2e380 R08: 0000000093808538 R09: 0000000000000000
R10: 0000000000000048 R11: 0000010419641fd8 R12: 0000010417abd2a0
R13: 0000000000000000 R14: 0000000000000001 R15: 00000103a13ade80
FS: 0000002a9589fb00(0000) GS:ffffffff80506f00(0000) knlGS:00000000f7e466c0
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000002a9556c000 CR3: 0000000819e2a000 CR4: 00000000000006e0
Process swapper (pid: 0, threadinfo 0000010419640000, task 0000010428056030)
Stack: 000000008c000000 ffffffffa00cc9a9 0000010307f1f518 0000010418204fd8
0000010417abd290 0000010417abd280 000005a800000000 8000000000000001
0000010418e2e000 0000010418e2e000
Call Trace:<IRQ> <ffffffffa00cc9a9>{:forcedeth:nv_start_xmit_optimized+991}
<ffffffff802c3699>{qdisc_restart+259} <ffffffff802d0797>{ip_finish_output2+0}
<ffffffff802b405f>{dev_queue_xmit+229} <ffffffff802bd976>{nf_hook_slow+184}
<ffffffff802d0b40>{ip_finish_output+496} <ffffffff802d0f02>{ip_queue_xmit+951}
<ffffffff80132789>{recalc_task_prio+337} <ffffffff80135db5>{autoremove_wake_function+9}
<ffffffff80132817>{activate_task+124} <ffffffff802e06d1>{tcp_transmit_skb+2064}
<ffffffff802e18e0>{tcp_retransmit_skb+1441} <ffffffff802e3b19>{tcp_write_timer+1059}
<ffffffff802e36f6>{tcp_write_timer+0} <ffffffff801406ed>{run_timer_softirq+356}
<ffffffff8013cd7c>{__do_softirq+88} <ffffffff8013ce25>{do_softirq+49}
<ffffffff80110c01>{apic_timer_interrupt+133} <EOI> <ffffffff8010e749>{default_idle+0}
<ffffffff8010e769>{default_idle+32} <ffffffff8010e7dc>{cpu_idle+26}
Code: 7e f9 e9 91 fd ff ff f3 90 83 3b 00 7e f9 e9 cf fd ff ff e8
Kernel panic - not syncing: nmi watchdog
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at panic:75
invalid operand: 0000 [1] SMP
CPU 11
Modules linked in: nfs lockd nfs_acl autofs4 sunrpc cpufreq_powersave ide_dump scsi_dump diskdump zlib_deflate dm_mod joydev button battery ac ohci_hcd ehci_hcd forcedeth ext3 jbd sata_nv libata mptscsih mptsas mptspi mptscsi mptbase sd_mod scsi_mod
Pid: 0, comm: swapper Not tainted 2.6.9-67.0.22.ELlargesmp
RIP: 0010:[<ffffffff80137fd2>] <ffffffff80137fd2>{panic+211}
RSP: 0018:0000010819e2cda8 EFLAGS: 00010086
RAX: 000000000000002c RBX: ffffffff80322f48 RCX: 0000000000000046
RDX: 0000000000025720 RSI: 0000000000000046 RDI: ffffffff803eb5c0
RBP: 0000010819e2cf58 R08: 0000000000000001 R09: ffffffff80322f48
R10: 0000000000000000 R11: 0000ffff80406a00 R12: 0000010417abd2a0
R13: 0000000000000000 R14: 0000000000000001 R15: 00000103a13ade80
FS: 0000002a9589fb00(0000) GS:ffffffff80506f00(0000) knlGS:00000000f7e466c0
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000002a9556c000 CR3: 0000000819e2a000 CR4: 00000000000006e0
Process swapper (pid: 0, threadinfo 0000010419640000, task 0000010428056030)
Stack: 0000003000000008 0000010819e2ce88 0000010819e2cdc8 0000000000000013
0000000000000000 0000000000000046 00000000000256f4 0000000000000046
0000000000000001 ffffffff803255be
Call Trace:<ffffffff80111870>{show_stack+241} <ffffffff8011199a>{show_registers+277}
<ffffffff80111ca1>{die_nmi+130} <ffffffff8011d1c2>{nmi_watchdog_tick+210}
<ffffffff8011256e>{default_do_nmi+112} <ffffffff8011d278>{do_nmi+115}
<ffffffff8011117f>{paranoid_exit+0} <ffffffff803100c3>{.text.lock.spinlock+49}
<EOE> <IRQ> <ffffffffa00cc9a9>{:forcedeth:nv_start_xmit_optimized+991}
<ffffffff802c3699>{qdisc_restart+259} <ffffffff802d0797>{ip_finish_output2+0}
<ffffffff802b405f>{dev_queue_xmit+229} <ffffffff802bd976>{nf_hook_slow+184}
<ffffffff802d0b40>{ip_finish_output+496} <ffffffff802d0f02>{ip_queue_xmit+951}
<ffffffff80132789>{recalc_task_prio+337} <ffffffff80135db5>{autoremove_wake_function+9}
<ffffffff80132817>{activate_task+124} <ffffffff802e06d1>{tcp_transmit_skb+2064}
<ffffffff802e18e0>{tcp_retransmit_skb+1441} <ffffffff802e3b19>{tcp_write_timer+1059}
<ffffffff802e36f6>{tcp_write_timer+0} <ffffffff801406ed>{run_timer_softirq+356}
<ffffffff8013cd7c>{__do_softirq+88} <ffffffff8013ce25>{do_softirq+49}
<ffffffff80110c01>{apic_timer_interrupt+133} <EOI> <ffffffff8010e749>{default_idle+0}
<ffffffff8010e769>{default_idle+32} <ffffffff8010e7dc>{cpu_idle+26}
Code: 0f 0b c8 34 32 80 ff ff ff ff 4b 00 31 ff e8 87 c2 fe ff e8
RIP <ffffffff80137fd2>{panic+211} RSP <0000010819e2cda8>
Environment
- Red Hat Enterprise Linux 4
- Red Hat Enterprise Linux 5
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
