RHEL7: mlx5 reports error messages during shutdown then panic with mce

Solution Verified - Updated -

Issue

  • Kernel panic with below logs:
[  200.824725] systemd-shutdown[1]: All filesystems unmounted.
[  200.824736] systemd-shutdown[1]: Deactivating swaps.
[  200.824837] systemd-shutdown[1]: All swaps deactivated.
[  200.824846] systemd-shutdown[1]: Detaching loop devices.
[  200.829664] systemd-shutdown[1]: All loop devices detached.
[  200.829674] systemd-shutdown[1]: Detaching DM devices.
[  200.829920] systemd-shutdown[1]: All DM devices detached.
[  200.848448] systemd-shutdown[1]: Syncing filesystems and block devices.
[  200.848779] systemd-shutdown[1]: Rebooting.
[  201.325192] infiniband mlx5_bond_0: wait_for_async_commands:745:(pid 3633): done with all pending requests
[  204.919209] mlx5_core 0000:af:00.1: Shutdown was called
[  205.014965] infiniband mlx5_bond_1: wait_for_async_commands:745:(pid 601): done with all pending requests
[  210.785156] infiniband mlx5_3: wait_for_async_commands:745:(pid 1): done with all pending requests
[  211.847192] mlx5_core 0000:af:00.0: Shutdown was called
[  215.000902] infiniband mlx5_2: wait_for_async_commands:745:(pid 1): done with all pending requests
[  215.310641] Disabling lock debugging due to kernel taint
[  215.310732] mce: [Hardware Error]: CPU 12: Machine Check Exception: 5 Bank 6: fb80000000000e0b
[  215.310733] mce: [Hardware Error]: Machine check events logged
[  215.311849] mce: [Hardware Error]: RIP !INEXACT! 10:<ffffffffb6b8e764> {intel_idle+0xd4/0x225}
[  215.312405] mce: [Hardware Error]: TSC 5405d7784ea MISC ae000000 
[  215.312956] mce: [Hardware Error]: PROCESSOR 0:50654 TIME 1650408701 SOCKET 1 APIC 40 microcode 2006c0a
[  215.313520] mce: [Hardware Error]: Run the above through 'mcelog --ascii'
[  215.333866] mce: [Hardware Error]: Machine check: Processor context corrupt
[  215.334472] Kernel panic - not syncing: Fatal machine check

Environment

  • Red Hat Enterprise Linux 7
  • HPE DL380 Gen10
  • Mellanox ConnectX-4 Lx
  • mlx5 fw 14.32.1010 (HP_2420110034)
  • mlx5 fw 14.31.1200 (HP_2420110034 / HP_2690110034)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content