Cicso UCS Rack server with Avago SAS Megaraid with an old server FW fails to boot RHEL 7.9 kernel

Solution Verified - Updated -

Issue

Cisco UCS Rack Server (UCSC-C3K-M4SRB) fails to boot vanilla RHEL 7.9 kernel with Avago SAS based MegaRAID fails to boot with the RHEL 7.9 kernel.

The boot sequence goes through normal SCSI subsystem initialisation, but eventually fails to access the root disk, then the boot goes to emergency shell with messages like these:

[   0.000000] Linux version 3.10.0-1160.el7.x86_64 (mockbuild@x86-vm-26.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 ( 
Red Hat 4.8.5-39) (GCC) ) #1 SMP Tue Aug 18 14:50:17 EDT 2020
    >>> vanilla RHEL 7.9 <<<
...
[  17.452275] megaraid_sas 0000:0c:00.0: FW provided supportMaxExtLDs: 1       max_lds: 64
[  17.452282] megaraid_sas 0000:0c:00.0: controller type       : MR(4095MB)
[  17.452286] megaraid_sas 0000:0c:00.0: Online Controller Reset(OCR)  : Enabled
[  17.452290] megaraid_sas 0000:0c:00.0: Secure JBOD support   : Yes
[  17.452293] megaraid_sas 0000:0c:00.0: NVMe passthru support : No
[  17.452297] megaraid_sas 0000:0c:00.0: FW provided TM TaskAbort/Reset timeout        : 0 secs/0 secs
[  17.452301] megaraid_sas 0000:0c:00.0: JBOD sequence map support     : Yes
[  17.452305] megaraid_sas 0000:0c:00.0: PCI Lane Margining support    : No
...
[  17.483330] megaraid_sas 0000:0c:00.0: megasas_enable_intr_fusion is called outbound_intr_mask:0x40000000
[  17.483334] megaraid_sas 0000:0c:00.0: INIT adapter done
[  17.491830] megaraid_sas 0000:0c:00.0: pci id                : (0x1000)/(0x00ce)/(0x1137)/(0x0197)
[  17.491842] megaraid_sas 0000:0c:00.0: unevenspan support    : no
[  17.491845] megaraid_sas 0000:0c:00.0: firmware crash dump   : no
[  17.491847] megaraid_sas 0000:0c:00.0: JBOD sequence map     : enabled
[  17.491907] scsi host0: Avago SAS based MegaRAID driver
[  17.498023] scsi 0:0:70:0: Enclosure         Cisco    C3260           2    PQ: 0 ANSI: 6
[  17.505838] fbcon: mgadrmfb (fb0) is primary device
[  17.513910] scsi 0:0:71:0: Enclosure         Cisco    C3260           2    PQ: 0 ANSI: 6
[  17.535966] scsi 0:2:0:0: Direct-Access     Cisco    UCS-C3K-M4RAID  4.70 PQ: 0 ANSI: 5
[  17.536314] scsi 0:2:1:0: Direct-Access     Cisco    UCS-C3K-M4RAID  4.70 PQ: 0 ANSI: 5
[  17.558303] scsi 0:2:2:0: Direct-Access     Cisco    UCS-C3K-M4RAID  4.70 PQ: 0 ANSI: 5
[  17.570304] scsi 0:2:3:0: Direct-Access     Cisco    UCS-C3K-M4RAID  4.70 PQ: 0 ANSI: 5
[  17.582450] scsi 0:2:4:0: Direct-Access     Cisco    UCS-C3K-M4RAID  4.70 PQ: 0 ANSI: 5
[  17.615320] scsi 0:2:5:0: Direct-Access     Cisco    UCS-C3K-M4RAID  4.70 PQ: 0 ANSI: 5
[  17.650163] sd 0:2:0:0: [sda] 935544832 512-byte logical blocks: (478 GB/446 GiB)
[  17.650163] sd 0:2:0:0: [sda] 4096-byte physical blocks
...
[  17.761215] mgag200 0000:0c:00.0: fb0: mgadrmfb frame buffer device
[  17.767324] [drm] initialized mgag200 1.0.0.20110418 for 0000:0c:00.0 on minor 0
...
[ 140.488033] ses 0:0:70:0: Attached Enclosure device
[ 140.488051] ses 0:0:71:0: Attached Enclosure device
... 
long gap of no messages?
...
[ 288.058052] sd 0:2:0:0: [sda] tag#10 CDB: Read(10) 28 00 01 80 0c 78 00 00 00 00
...
[ 289.852938] megaraid_sas 0000:0c:00.0: Failed from megasas_fault_detect_work 1921, do not rearm timer
...
[ 301.470085] sd 0:2:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK cmd_age=0s
[ 301.470095] sd 0:2:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[ 301.470100] blk_update_request: I/O error, dev_sda, sector 0
...

The same HW configuration is booting an older kernel version 3.10.0-862.el7.x86_64 without any problems.

Environment

  • Cisco UCS Rack Server (Product UCSC-C3K-M4SRB)
  • BIOS version C3X60M4.3.0.3b.0.0325171543
  • root disk accessed via Broadcom / LSI MegaRAID SAS-3 3316 [Intruder] [1000:00ce] (rev 01)
  • booting RHEL 7.9 kernel 3.10.0-1160.el7.x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content