During VM migration in OpenShift Virtualization, multipath paths drop to 'failed faulty running' on HPE Alletra (Nimble) FC storage arrays
Issue
When migrating a VM hosted by OpenShift Virtualization using HPE Alletra 5000/6000 (Nimble) Storage Array(s) will fail with an IO Error.
Before the Migration:
# multipath -ll
mpathd (256a16a3ba7xxxxxxxxxxx0e8bdcb) dm-0 Nimble,Server
size=60G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 1:0:0:1 sda 8:0 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
`- 2:0:0:1 sdb 8:16 active ready running
During/after the Migration attempt:
The mpath device may be different, but it will show the state failed faulty running on the former hypervisor and on the new hypervisor:
mpathd (256a16a3ba7xxxxxxxxxxx0e8bdcb) dm-0 Nimble,Server
size=60G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 1:0:0:1 sda 8:0 failed faulty running
`-+- policy='service-time 0' prio=10 status=enabled
`- 2:0:0:1 sdb 8:16 failed faulty running
The dmesg logs from the target hypervisor report Power-on or device reset occurred messages:
Apr 25 16:28:41 2025] scsi 1:0:0:0: Direct-Access Nimble Server 1.0 PQ: 0 ANSI: 5
[Fri Apr 25 16:28:41 2025] scsi 1:0:0:0: alua: supports implicit TPGS
[Fri Apr 25 16:28:41 2025] scsi 1:0:0:0: alua: device eui.56a16a3ba79xxxxxxxxxxx0e8bdcb port group 1 rel port 3
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: Attached scsi generic sg2 type 0
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.
[Fri Apr 25 16:28:41 2025] scsi 2:0:0:0: Direct-Access Nimble Server 1.0 PQ: 0 ANSI: 5
[Fri Apr 25 16:28:41 2025] scsi 2:0:0:0: alua: supports implicit TPGS
[Fri Apr 25 16:28:41 2025] scsi 2:0:0:0: alua: device eui.56a16a3ba79xxxxxxZxxxx0e8bdcb port group 2 rel port 7
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: Attached scsi generic sg3 type 0
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: Power-on or device reset occurred
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: alua: transition timeout set to 60 seconds
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: alua: port group 01 state A non-preferred supports tolusna
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] 125829120 512-byte logical blocks: (64.4 GB/60.0 GiB)
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] Write Protect is off
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] Mode Sense: 9b 00 00 08
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: alua: transition timeout set to 60 seconds
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: alua: port group 02 state N non-preferred supports tolusna
Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] 125829120 512-byte logical blocks: (64.4 GB/60.0 GiB)
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] Write Protect is off
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] Mode Sense: 9b 00 00 08
[Fri Apr 25 16:28:41 2025] sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: alua: transition timeout set to 60 seconds
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: alua: port group 02 state N non-preferred supports tolusna
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: [sdc] 125829120 512-byte logical blocks: (64.4 GB/60.0 GiB)
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: [sdc] Write Protect is off
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: [sdc] Mode Sense: 9b 00 00 08
[Fri Apr 25 16:28:41 2025] sd 2:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
<This will repeat>
Environment
- OpenShift-Virtualization
VERSION="417.94.202504012217-0"
VERSION_ID="4.17" (can happen at any version at this time)
VARIANT="CoreOS" - HPE Alletra 5000/6000 storage array(s)
- RHEL and/or Windows VMs
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.