RHEL6.2 Link is Down happens irregularly by INTEL NIC X540.

Solution Unverified - Updated -

Issue

  • INTEL 10GNIC of customer's BOX irregularly downs Link.
  • The temperature message is recorded and inform this message of known issue about the relation.
Jul  6 06:30:55 localhost kernel: [Hardware Error]: Machine check events logged
Jul  6 06:30:55 localhost mcelog: Processor 20 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 4 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 4 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 20 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 0 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 2 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 18 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 6 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 22 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 8 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 24 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 10 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 26 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 28 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 12 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 30 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 14 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 0 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 2 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 18 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 6 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 22 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 8 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 24 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 10 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 26 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 28 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 12 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 30 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 14 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 16 below trip temperature. Throttling disabled
Jul  6 06:30:55 localhost mcelog: Processor 16 below trip temperature. Throttling disabled
Jul  6 06:55:53 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Down
Jul  6 06:55:57 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
Jul  6 06:55:58 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Down
Jul  6 06:55:59 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
Jul  6 06:55:59 localhost kernel: connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4350537676, last ping 4350542676, now 4350547676
Jul  6 06:55:59 localhost kernel: connection1:0: detected conn error (1011)
Jul  6 06:55:59 localhost kernel: connection4:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4350537678, last ping 4350542678, now 4350547679
Jul  6 06:55:59 localhost kernel: connection4:0: detected conn error (1011)
Jul  6 06:55:59 localhost kernel: connection2:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4350537679, last ping 4350542679, now 4350547679
Jul  6 06:55:59 localhost kernel: connection2:0: detected conn error (1011)
Jul  6 06:56:00 localhost iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3)
Jul  6 06:56:00 localhost iscsid: Kernel reported iSCSI connection 4:0 error (1011) state (3)
Jul  6 06:56:00 localhost iscsid: Kernel reported iSCSI connection 2:0 error (1011) state (3)
Jul  6 06:56:02 localhost kernel: connection1:0: detected conn error (1020)
Jul  6 06:56:02 localhost kernel: connection4:0: detected conn error (1020)
Jul  6 06:56:02 localhost kernel: connection2:0: detected conn error (1020)
Jul  6 06:56:03 localhost iscsid: connection1:0 is operational after recovery (1 attempts)
Jul  6 06:56:03 localhost iscsid: connection4:0 is operational after recovery (1 attempts)
Jul  6 06:56:03 localhost iscsid: connection2:0 is operational after recovery (1 attempts)
Jul  6 07:33:12 localhost kernel: CPU4: Core power limit notification (total events = 136)
Jul  6 07:33:12 localhost kernel: CPU20: Core power limit notification (total events = 136)
Jul  6 07:33:12 localhost kernel: CPU20: Package power limit notification (total events = 136)
Jul  6 07:33:12 localhost kernel: CPU4: Package power limit notification (total events = 136)

Environment

  • Red Hat Enterprise Linux 6.2 Server
  • Architecture:x86_64
  • Kernel Version:2.6.32-220.23.1.el6.x86_64
  • Related Package Version:
  • Related Middleware/Application: Oracle RAC
  • Drivers or hardware or archtecture dependency: It happens by both the default driver and DELL.
  • Intel(R) 10 Gigabit PCI Express Network Driver - version 3.7.21-NAP

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content