RHEL6.2 Link is Down happens irregularly by INTEL NIC X540.
Issue
- INTEL 10GNIC of customer's BOX irregularly downs Link.
- The temperature message is recorded and inform this message of known issue about the relation.
Jul 6 06:30:55 localhost kernel: [Hardware Error]: Machine check events logged
Jul 6 06:30:55 localhost mcelog: Processor 20 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 4 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 4 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 20 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 0 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 2 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 18 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 6 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 22 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 8 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 24 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 10 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 26 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 28 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 12 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 30 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 14 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 0 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 2 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 18 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 6 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 22 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 8 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 24 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 10 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 26 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 28 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 12 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 30 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 14 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 16 below trip temperature. Throttling disabled
Jul 6 06:30:55 localhost mcelog: Processor 16 below trip temperature. Throttling disabled
Jul 6 06:55:53 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Down
Jul 6 06:55:57 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
Jul 6 06:55:58 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Down
Jul 6 06:55:59 localhost kernel: ixgbe 0000:01:00.0: em1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
Jul 6 06:55:59 localhost kernel: connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4350537676, last ping 4350542676, now 4350547676
Jul 6 06:55:59 localhost kernel: connection1:0: detected conn error (1011)
Jul 6 06:55:59 localhost kernel: connection4:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4350537678, last ping 4350542678, now 4350547679
Jul 6 06:55:59 localhost kernel: connection4:0: detected conn error (1011)
Jul 6 06:55:59 localhost kernel: connection2:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4350537679, last ping 4350542679, now 4350547679
Jul 6 06:55:59 localhost kernel: connection2:0: detected conn error (1011)
Jul 6 06:56:00 localhost iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3)
Jul 6 06:56:00 localhost iscsid: Kernel reported iSCSI connection 4:0 error (1011) state (3)
Jul 6 06:56:00 localhost iscsid: Kernel reported iSCSI connection 2:0 error (1011) state (3)
Jul 6 06:56:02 localhost kernel: connection1:0: detected conn error (1020)
Jul 6 06:56:02 localhost kernel: connection4:0: detected conn error (1020)
Jul 6 06:56:02 localhost kernel: connection2:0: detected conn error (1020)
Jul 6 06:56:03 localhost iscsid: connection1:0 is operational after recovery (1 attempts)
Jul 6 06:56:03 localhost iscsid: connection4:0 is operational after recovery (1 attempts)
Jul 6 06:56:03 localhost iscsid: connection2:0 is operational after recovery (1 attempts)
Jul 6 07:33:12 localhost kernel: CPU4: Core power limit notification (total events = 136)
Jul 6 07:33:12 localhost kernel: CPU20: Core power limit notification (total events = 136)
Jul 6 07:33:12 localhost kernel: CPU20: Package power limit notification (total events = 136)
Jul 6 07:33:12 localhost kernel: CPU4: Package power limit notification (total events = 136)
Environment
- Red Hat Enterprise Linux 6.2 Server
- Architecture:x86_64
- Kernel Version:2.6.32-220.23.1.el6.x86_64
- Related Package Version:
- Related Middleware/Application: Oracle RAC
- Drivers or hardware or archtecture dependency: It happens by both the default driver and DELL.
- Intel(R) 10 Gigabit PCI Express Network Driver - version 3.7.21-NAP
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.